Hyperledger Fabric - Fail to add orderer to a channel with v2.3 version. cannot join: failed to determine cluster membership from join-block

Viewed 488

I’m trying to create a channel with hyperledger fabric v2.3 (no system channel) fallowing docs for creating a channel. And I failed to add first orderer to a channel. Please tell me what makes the error.

Create genesis block successfully:

configtxgen -profile AMLChannelGenesis -outputBlock ./channel-artifacts/genesis_block.pb -channelID amlchannel
2021-05-23 03:37:23.585 UTC [common.tools.configtxgen] doOutputBlock -> INFO 05c Generating genesis block
2021-05-23 03:37:23.585 UTC [common.tools.configtxgen] doOutputBlock -> INFO 05d Creating application channel genesis block
2021-05-23 03:37:23.586 UTC [common.tools.configtxgen] doOutputBlock -> INFO 05e Writing genesis block

then:

export OSN_TLS_CA_ROOT_CERT=/tmp/hyperledger/org0/orderer/tls-msp/tlscacerts/tls-0-0-0-0-7052.pem 
export ADMIN_TLS_SIGN_CERT=/tmp/hyperledger/org0/orderer/tls-msp/signcerts/cert.pem 
export ADMIN_TLS_PRIVATE_KEY=/tmp/hyperledger/org0/orderer/tls-msp/keystore/key.pem 
osnadmin channel join --channelID amlchannel --config-block /tmp/hyperledger/configtx/channel-artifacts/genesis_block.pb -o orderer1-org0:8050 --ca-file $OSN_TLS_CA_ROOT_CERT --client-cert $ADMIN_TLS_SIGN_CERT --client-key $ADMIN_TLS_PRIVATE_KEY 

but can't join channel:

Status: 400
{
    "error": "cannot join: failed to determine cluster membership from join-block: failed to validate config metadata of ordering config: verifying tls client cert with serial number 384514493998453233869746683140128894521987234218: x509: certificate signed by unknown authority"
}

orderer docker-compose:

orderer1-org0:
    container_name: orderer1-org0
    image: hyperledger/fabric-orderer:2.3
    environment:
      - ORDERER_HOME=/tmp/hyperledger/orderer
      - ORDERER_HOST=orderer1-org0
      - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
      - ORDERER_GENERAL_LISTENPORT=7050
      - ORDERER_GENERAL_TLS_ENABLED=true

      - ORDERER_GENERAL_TLS_PRIVATEKEY=/tmp/hyperledger/org0/orderer/tls-msp/keystore/key.pem
      - ORDERER_GENERAL_TLS_CERTIFICATE=/tmp/hyperledger/org0/orderer/tls-msp/signcerts/cert.pem
      - ORDERER_GENERAL_TLS_ROOTCAS=[/tmp/hyperledger/org0/orderer/tls-msp/tlscacerts/tls-0-0-0-0-7052.pem]

      - ORDERER_KAFKA_TOPIC_REPLICATIONFACTOR=1
      - ORDERER_KAFKA_VERBOSE=true

      - ORDERER_GENERAL_CLUSTER_CLIENTCERTIFICATE=/tmp/hyperledger/org0/orderer/tls-msp/signcerts/cert.pem
      - ORDERER_GENERAL_CLUSTER_CLIENTPRIVATEKEY=/tmp/hyperledger/org0/orderer/tls-msp/keystore/key.pem
      - ORDERER_GENERAL_CLUSTER_ROOTCAS=[/tmp/hyperledger/org0/orderer/tls-msp/tlscacerts/tls-0-0-0-0-7052.pem]

      - ORDERER_GENERAL_LOCALMSPID=org0MSP
      - ORDERER_GENERAL_LOCALMSPDIR=/tmp/hyperledger/org0/orderer/msp
      - ORDERER_GENERAL_LOGLEVEL=debug
      - ORDERER_DEBUG_BROADCASTTRACEDIR=data/logs

      - FABRIC_LOGGING_SPEC=INFO

      - ORDERER_GENERAL_BOOTSTRAPMETHOD=none
      - ORDERER_CHANNELPARTICIPATION_ENABLED=true
      - ORDERER_ADMIN_TLS_ENABLED=true

      - ORDERER_ADMIN_TLS_CERTIFICATE=/tmp/hyperledger/org0/orderer/tls-msp/signcerts/cert.pem
      - ORDERER_ADMIN_TLS_PRIVATEKEY=/tmp/hyperledger/org0/orderer/tls-msp/keystore/key.pem
      - ORDERER_ADMIN_TLS_ROOTCAS=[/tmp/hyperledger/org0/orderer/tls-msp/tlscacerts/tls-0-0-0-0-7052.pem]
      - ORDERER_ADMIN_TLS_CLIENTROOTCAS=[/tmp/hyperledger/org0/orderer/tls-msp/tlscacerts/tls-0-0-0-0-7052.pem]
      
      - ORDERER_ADMIN_TLS_CLIENTAUTHREQUIRED=true
      - ORDERER_ADMIN_LISTENADDRESS=0.0.0.0:8050

    volumes:
      - /home/ted/AML_fabricTest/org0/orderer:/tmp/hyperledger/org0/orderer/
      - /home/ted/AML_fabricTest/configtx/system-genesis-block/aml_genesis.block:/tmp/hyperledger/orderer/orderer.genesis.block
    
    ports:
      - 8050:8050
    
    networks:
      - fabric-ca

osnadmin cli docker-compose (create genesis block and join channel in this docker):

cli-osnadmin:
      container_name: cli-osnadmin
      image: hyperledger/fabric-tools:2.3
      tty: true
      stdin_open: true
      environment:
        - GOPATH=/opt/gopath
        - CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
        - FABRIC_LOGGING_SPEC=DEBUG
      working_dir: /opt/gopath/src/github.com/hyperledger/fabric
      command: /bin/bash
      volumes:
        - /home/ted/AML_fabricTest/org0:/tmp/hyperledger/org0/
        - /home/ted/AML_fabricTest/org1:/tmp/hyperledger/org1/
        - /home/ted/AML_fabricTest/org2:/tmp/hyperledger/org2/
        - /home/ted/AML_fabricTest/fabric-ca-tls/admin:/tmp/hyperledger/admin-client/
        - /home/ted/AML_fabricTest/configtx/:/tmp/hyperledger/configtx
      networks:
        - fabric-ca        

configtx:

# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

---
################################################################################
#
#   Section: Organizations
#
#   - This section defines the different organizational identities which will
#   be referenced later in the configuration.
#
################################################################################
Organizations:

    # SampleOrg defines an MSP using the sampleconfig.  It should never be used
    # in production but may be used as a template for other definitions
    - &org0-orderer
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
        Name: org0-ordererMSP
        SkipAsForeign: false
        # ID to load the MSP definition as
        ID: org0-ordererMSP

        # MSPDir is the filesystem path which contains the MSP configuration
        MSPDir: /tmp/hyperledger/org0/orderer/msp

        # Policies defines the set of policies at this level of the config tree
        # For organization policies, their canonical path is usually
        #   /Channel/<Application|Orderer>/<OrgName>/<PolicyName>
        Policies:
            Readers:
                Type: Signature
                Rule: "OR('org0-ordererMSP.member')"
            Writers:
                Type: Signature
                Rule: "OR('org0-ordererMSP.member')"
            Admins:
                Type: Signature
                Rule: "OR('org0-ordererMSP.admin')"

        OrdererEndpoints:
            - orderer1-org0:7050
    - &org0
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
        Name: org0MSP

        # ID to load the MSP definition as
        ID: org0MSP

        # MSPDir is the filesystem path which contains the MSP configuration
        MSPDir: /tmp/hyperledger/org0/peer1/msp

        # Policies defines the set of policies at this level of the config tree
        # For organization policies, their canonical path is usually
        #   /Channel/<Application|Orderer>/<OrgName>/<PolicyName>
        Policies:
            Readers:
                Type: Signature
                Rule: "OR('org0MSP.member')"
            Writers:
                Type: Signature
                Rule: "OR('org0MSP.member')"
            Admins:
                Type: Signature
                Rule: "OR('org0MSP.admin')"
            Endorsement:
                Type: Signature
                Rule: "OR('org0MSP.peer')"
        # leave this flag set to true.
        AnchorPeers:
            # AnchorPeers defines the location of peers which can be used
            # for cross org gossip communication.  Note, this value is only
            # encoded in the genesis block in the Application section context
            - Host: peer1-org0
              Port: 7051

    - &org1-orderer
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
        Name: org1-ordererMSP
        SkipAsForeign: false
        # ID to load the MSP definition as
        ID: org1-ordererMSP

        MSPDir: /tmp/hyperledger/org1/orderer/msp

        # Policies defines the set of policies at this level of the config tree
        # For organization policies, their canonical path is usually
        #   /Channel/<Application|Orderer>/<OrgName>/<PolicyName>
        Policies:
            Readers:
                Type: Signature
                Rule: "OR('org1-ordererMSP.member')"
            Writers:
                Type: Signature
                Rule: "OR('org1-ordererMSP.member')"
            Admins:
                Type: Signature
                Rule: "OR('org1-ordererMSP.admin')"

        OrdererEndpoints:
            - orderer1-org1:7050

    - &org1
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
        Name: org1MSP

        # ID to load the MSP definition as
        ID: org1MSP

        MSPDir: /tmp/hyperledger/org1/peer1/msp

        # Policies defines the set of policies at this level of the config tree
        # For organization policies, their canonical path is usually
        #   /Channel/<Application|Orderer>/<OrgName>/<PolicyName>
        Policies:
            Readers:
                Type: Signature
                Rule: "OR('org1MSP.admin', 'org1MSP.peer', 'org1MSP.client')"
            Writers:
                Type: Signature
                Rule: "OR('org1MSP.admin', 'org1MSP.client')"
            Admins:
                Type: Signature
                Rule: "OR('org1MSP.admin')"
            Endorsement:
                Type: Signature
                Rule: "OR('org1MSP.peer')"

        # leave this flag set to true.
        AnchorPeers:
            # AnchorPeers defines the location of peers which can be used
            # for cross org gossip communication.  Note, this value is only
            # encoded in the genesis block in the Application section context
            - Host: peer1-org1
              Port: 7051

    - &org2-orderer
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
        Name: org2-ordererMSP
        SkipAsForeign: false
        # ID to load the MSP definition as
        ID: org2-ordererMSP

        MSPDir: /tmp/hyperledger/org2/orderer/msp

        # Policies defines the set of policies at this level of the config tree
        # For organization policies, their canonical path is usually
        #   /Channel/<Application|Orderer>/<OrgName>/<PolicyName>
        Policies:
            Readers:
                Type: Signature
                Rule: "OR('org2-ordererMSP.member')"
            Writers:
                Type: Signature
                Rule: "OR('org2-ordererMSP.member')"
            Admins:
                Type: Signature
                Rule: "OR('org2-ordererMSP.admin')"

        OrdererEndpoints:
            - orderer1-org2:7050

    - &org2
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
        Name: org2MSP

        # ID to load the MSP definition as
        ID: org2MSP

        MSPDir: /tmp/hyperledger/org2/peer1/msp

        # Policies defines the set of policies at this level of the config tree
        # For organization policies, their canonical path is usually
        #   /Channel/<Application|Orderer>/<OrgName>/<PolicyName>
        Policies:
            Readers:
                Type: Signature
                Rule: "OR('org2MSP.admin', 'org2MSP.peer', 'org2MSP.client')"
            Writers:
                Type: Signature
                Rule: "OR('org2MSP.admin', 'org2MSP.client')"
            Admins:
                Type: Signature
                Rule: "OR('org2MSP.admin')"
            Endorsement:
                Type: Signature
                Rule: "OR('org2MSP.peer')"

        AnchorPeers:
            # AnchorPeers defines the location of peers which can be used
            # for cross org gossip communication.  Note, this value is only
            # encoded in the genesis block in the Application section context
            - Host: peer1-org2
              Port: 7051
################################################################################
#
#   SECTION: Capabilities
#
#   - This section defines the capabilities of fabric network. This is a new
#   concept as of v1.1.0 and should not be utilized in mixed networks with
#   v1.0.x peers and orderers.  Capabilities define features which must be
#   present in a fabric binary for that binary to safely participate in the
#   fabric network.  For instance, if a new MSP type is added, newer binaries
#   might recognize and validate the signatures from this type, while older
#   binaries without this support would be unable to validate those
#   transactions.  This could lead to different versions of the fabric binaries
#   having different world states.  Instead, defining a capability for a channel
#   informs those binaries without this capability that they must cease
#   processing transactions until they have been upgraded.  For v1.0.x if any
#   capabilities are defined (including a map with all capabilities turned off)
#   then the v1.0.x peer will deliberately crash.
#
################################################################################
Capabilities:
    # Channel capabilities apply to both the orderers and the peers and must be
    # supported by both.
    # Set the value of the capability to true to require it.
    Channel: &ChannelCapabilities
        # V2_0 capability ensures that orderers and peers behave according
        # to v2.0 channel capabilities. Orderers and peers from
        # prior releases would behave in an incompatible way, and are therefore
        # not able to participate in channels at v2.0 capability.
        # Prior to enabling V2.0 channel capabilities, ensure that all
        # orderers and peers on a channel are at v2.0.0 or later.
        V2_0: true

    # Orderer capabilities apply only to the orderers, and may be safely
    # used with prior release peers.
    # Set the value of the capability to true to require it.
    Orderer: &OrdererCapabilities
        # V2_0 orderer capability ensures that orderers behave according
        # to v2.0 orderer capabilities. Orderers from
        # prior releases would behave in an incompatible way, and are therefore
        # not able to participate in channels at v2.0 orderer capability.
        # Prior to enabling V2.0 orderer capabilities, ensure that all
        # orderers on channel are at v2.0.0 or later.
        V2_0: true

    # Application capabilities apply only to the peer network, and may be safely
    # used with prior release orderers.
    # Set the value of the capability to true to require it.
    Application: &ApplicationCapabilities
        # V2_0 application capability ensures that peers behave according
        # to v2.0 application capabilities. Peers from
        # prior releases would behave in an incompatible way, and are therefore
        # not able to participate in channels at v2.0 application capability.
        # Prior to enabling V2.0 application capabilities, ensure that all
        # peers on channel are at v2.0.0 or later.
        V2_0: true

################################################################################
#
#   SECTION: Application
#
#   - This section defines the values to encode into a config transaction or
#   genesis block for application related parameters
#
################################################################################
Application: &ApplicationDefaults

    # Organizations is the list of orgs which are defined as participants on
    # the application side of the network
    Organizations:

    # Policies defines the set of policies at this level of the config tree
    # For Application policies, their canonical path is
    #   /Channel/Application/<PolicyName>
    Policies:
        Readers:
            Type: ImplicitMeta
            Rule: "ANY Readers"
        Writers:
            Type: ImplicitMeta
            Rule: "ANY Writers"
        Admins:
            Type: ImplicitMeta
            Rule: "MAJORITY Admins"
        LifecycleEndorsement:
            Type: ImplicitMeta
            Rule: "MAJORITY Endorsement"
        Endorsement:
            Type: ImplicitMeta
            Rule: "MAJORITY Endorsement"

    Capabilities:
        <<: *ApplicationCapabilities
################################################################################
#
#   SECTION: Orderer
#
#   - This section defines the values to encode into a config transaction or
#   genesis block for orderer related parameters
#
################################################################################
Orderer: &OrdererDefaults

    # Orderer Type: The orderer implementation to start
    OrdererType: etcdraft

    EtcdRaft:
        Consenters:
        - Host: orderer1-org0
          Port: 7050
          ClientTLSCert: /tmp/hyperledger/org0/orderer/tls-msp/signcerts/cert.pem
          ServerTLSCert: /tmp/hyperledger/org0/orderer/tls-msp/signcerts/cert.pem
        - Host: orderer1-org1
          Port: 7050
          ClientTLSCert: /tmp/hyperledger/org1/orderer/tls-msp/signcerts/cert.pem
          ServerTLSCert: /tmp/hyperledger/org1/orderer/tls-msp/signcerts/cert.pem
        - Host: orderer1-org2
          Port: 7050
          ClientTLSCert: /tmp/hyperledger/org2/orderer/tls-msp/signcerts/cert.pem
          ServerTLSCert: /tmp/hyperledger/org2/orderer/tls-msp/signcerts/cert.pem

    # Batch Timeout: The amount of time to wait before creating a batch
    BatchTimeout: 2s

    # Batch Size: Controls the number of messages batched into a block
    BatchSize:

        # Max Message Count: The maximum number of messages to permit in a batch
        MaxMessageCount: 10

        # Absolute Max Bytes: The absolute maximum number of bytes allowed for
        # the serialized messages in a batch.
        AbsoluteMaxBytes: 99 MB

        # Preferred Max Bytes: The preferred maximum number of bytes allowed for
        # the serialized messages in a batch. A message larger than the preferred
        # max bytes will result in a batch larger than preferred max bytes.
        PreferredMaxBytes: 512 KB

    # Organizations is the list of orgs which are defined as participants on
    # the orderer side of the network
    Organizations:

    # Policies defines the set of policies at this level of the config tree
    # For Orderer policies, their canonical path is
    #   /Channel/Orderer/<PolicyName>
    Policies:
        Readers:
            Type: ImplicitMeta
            Rule: "ANY Readers"
        Writers:
            Type: ImplicitMeta
            Rule: "ANY Writers"
        Admins:
            Type: ImplicitMeta
            Rule: "MAJORITY Admins"
        # BlockValidation specifies what signatures must be included in the block
        # from the orderer for the peer to validate it.
        BlockValidation:
            Type: ImplicitMeta
            Rule: "ANY Writers"

################################################################################
#
#   CHANNEL
#
#   This section defines the values to encode into a config transaction or
#   genesis block for channel related parameters.
#
################################################################################
Channel: &ChannelDefaults
    # Policies defines the set of policies at this level of the config tree
    # For Channel policies, their canonical path is
    #   /Channel/<PolicyName>
    Policies:
        # Who may invoke the 'Deliver' API
        Readers:
            Type: ImplicitMeta
            Rule: "ANY Readers"
        # Who may invoke the 'Broadcast' API
        Writers:
            Type: ImplicitMeta
            Rule: "ANY Writers"
        # By default, who may modify elements at this config level
        Admins:
            Type: ImplicitMeta
            Rule: "MAJORITY Admins"

    # Capabilities describes the channel level capabilities, see the
    # dedicated Capabilities section elsewhere in this file for a full
    # description
    Capabilities:
        <<: *ChannelCapabilities

################################################################################
#
#   Profile
#
#   - Different configuration profiles may be encoded here to be specified
#   as parameters to the configtxgen tool
#
################################################################################
Profiles:
    AMLChannelGenesis:
        <<: *ChannelDefaults
        Orderer:
            <<: *OrdererDefaults
            OrdererType: etcdraft
            Organizations:
                - *org0-orderer
                - *org1-orderer
                - *org2-orderer
            Capabilities:
                <<: *OrdererCapabilities
        Application:
            <<: *ApplicationDefaults
            Organizations:
                - *org0
                - *org1
                - *org2
            Capabilities:
                <<: *ApplicationCapabilities     

My file structure:

├── fabric
│       ├── org0
│       │   ├── orderer
│       │   │   ├── msp
│       │   │   │   ├── IssuerPublicKey
│       │   │   │   ├── IssuerRevocationPublicKey
│       │   │   │   ├── config.yaml
│       │   │   │   ├── admincerts
│       │   │   │   │ └── orderer-admin-cert.pem
│       │   │   │   ├── cacerts
│       │   │   │   │ └── ca-cert.pem
│       │   │   │   ├── keystore
│       │   │   │   │ └── key.pem
│       │   │   │   ├── signcerts
│       │   │   │   │ └── cert.pem
│       │   │   │   └── user
│       │   │   └── tls
│       │   │       ├── IssuerPublicKey
│       │   │       ├── IssuerRevocationPublicKey
│       │   │       ├── config.yaml
│       │   │       ├── cacerts
│       │   │       ├── tlscacerts
│       │   │       │   └── tls-ca-cert.pem
│       │   │       ├── keystore
│       │   │       │   └── key.pem
│       │   │       ├── signcerts
│       │   │       │   └── cert.pem
│       │   │       └── user
│       │   ├── peer(...)
│       │   └── admin(...) 
...

I try several days with multiple --ca-file / --client-cert / --client-key variable, but still can not join channel.

but I can get node information by

osnadmin channel list -o orderer1-org0:8050 --ca-file $OSN_TLS_CA_ROOT_CERT --client-cert $ADMIN_TLS_SIGN_CERT --client-key $ADMIN_TLS_PRIVATE_KEY

and get:

Status: 200
{
    "systemChannel": null,
    "channels": null
}
1 Answers

Your orderer orderer1-org0 MSPID is org0MSP and all the credentials are generated under this MSPID but this MSPID is not in the configtx: profile.AMLChannelGenesis.Orderer.Organizations so the orderer orderer1-org0 is not valid for joining the channel as an orderer. (although it is in the Consenters list)

try adding org0MSP to configtx: profile.AMLChannelGenesis.Orderer.Organizations.

or create your orderer credentials under org0-ordererMSP

Related