Why is terraform pulling a provider from Hashicorp I declare nowhere?

user_889053 0
Viewed 1804

I am trying to test out some new code for a terraform provider and for reasons I don't understand, it seems to want to try and find a resource definition with hashicorp, even though it's specifically defined within this provider. Obviously I am missing something:

Code to [provider][1]:

This is how I build and install it:

christianb@unifi terraform-provider-artifactory % rm -fR .terraform                                                               
christianb@unifi terraform-provider-artifactory % go build && mkdir -p .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64 && mv terraform-provider-artifactory .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64

now running it

christianb@unifi terraform-provider-artifactory % TF_LOG=trace terraform init
2021/02/25 13:12:56 [INFO] Terraform version: 0.13.5  
2021/02/25 13:12:56 [INFO] Go runtime version: go1.14.10
2021/02/25 13:12:56 [INFO] CLI args: []string{"/usr/local/bin/terraform", "init"}
2021/02/25 13:12:56 [DEBUG] Attempting to open CLI config file: /Users/christianb/.terraformrc
2021/02/25 13:12:56 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory /Users/christianb/.terraform.d/plugins
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory /Users/christianb/Library/Application Support/io.terraform/plugins
2021/02/25 13:12:56 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2021/02/25 13:12:56 [INFO] CLI command args: []string{"init"}

Initializing the backend...
2021/02/25 13:12:56 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2021/02/25 13:12:56 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2021/02/25 13:12:56 [DEBUG] New state was assigned lineage "b7f7e5b9-b88c-6195-aaaf-e38b3008b8e5"
2021/02/25 13:12:56 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2021/02/25 13:12:56 [TRACE] Meta.Backend: instantiated backend of type <nil>
2021/02/25 13:12:56 [DEBUG] checking for provisioner in "."
2021/02/25 13:12:56 [DEBUG] checking for provisioner in "/usr/local/bin"
2021/02/25 13:12:56 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2021/02/25 13:12:56 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2021/02/25 13:12:56 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2021/02/25 13:12:56 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2021/02/25 13:12:56 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2021/02/25 13:12:56 [TRACE] statemgr.Filesystem: read nil snapshot

2021/02/25 13:12:56 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2021/02/25 13:12:56 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
Initializing provider plugins...
2021/02/25 13:12:56 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/jfrog/artifactory v2.2.6-alpha for darwin_amd64 at .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64
2021/02/25 13:12:56 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/jfrog/artifactory/2.2.6-alpha/darwin_amd64 as a candidate package for registry.terraform.io/jfrog/artifactory 2.2.6-alpha
2021/02/25 13:12:56 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2021/02/25 13:12:56 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
- Using previously-installed jfrog/artifactory v2.2.6-alpha
- Finding latest version of hashicorp/xray...
2021/02/25 13:12:56 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/xray/versions
2021/02/25 13:12:56 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/xray/versions
2021/02/25 13:12:57 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
2021/02/25 13:12:57 [DEBUG] GET https://registry.terraform.io/v1/providers/-/xray/versions
2021/02/25 13:12:57 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/-/xray/versions

Error: Failed to install provider

Error while installing hashicorp/xray: provider registry registry.terraform.io
does not have a provider named registry.terraform.io/hashicorp/xray

terraform to run:

# Required for Terraform 0.13 and up (https://www.terraform.io/upgrade-guides/0-13.html)
terraform {
  required_providers {
    artifactory = {
      source  = "registry.terraform.io/jfrog/artifactory"
      version = "2.2.6-alpha"
    }
  }
}
provider "artifactory" {
  url = "http://localhost:8082/artifactory"
  username = "admin"
  password = "password"
}

resource "xray_policy" "test" {
  name  = "test-policy-name"
  description = "test policy description"
  type = "security"

  rules {
    name = "rule-name"
    priority = 1
    criteria {
      min_severity = "High"
    }
    actions {
      block_download {
        unscanned = true
        active = true
      }
    }
  }
}

resource "xray_watch" "test" {
  name = "test-watch-name"
  description = "test watch description"
  resources {
    type = "all-repos"
    name = "All Repositories"
  }
  assigned_policies {
    name = xray_policy.test.name
    type = "security"
  }
  watch_recipients = ["test@example.com"]
}

I figured out it's based on the name of the resource type - so, that it doesn't start with artifactory makes a difference. What I'd like to know is the logic behind this lookup [1]: https://github.com/ryndaniels/terraform-provider-xray

terraform
1 Answers

Terraform v0.13 introduced the idea of third-party providers that belong to other namespaces that are not controlled directly by HashiCorp, but to maximize backward compatibility with modules that were written for Terraform v0.12 and earlier there is a fallback behavior where Terraform will assume that any provider requirement not explicitly declared is aiming to use one of the official providers which now live in the "hashicorp" namespace in the registry, because for Terraform v0.12 and earlier third-party providers were not automatically installable at all.

When writing modules for Terraform v0.13 or later you should include explicit provider requirements to specify the full source addresses for each of the providers your module uses, like this:

terraform {
  required_providers {
    artifactory = {
      source = "jfrog/artifactory"
    }
    xray = {
      source = "ryndaniels/xray"
    }
  }
}

Unfortunately the "xray" provider you are trying to use here hasn't, at the time of my writing this, been published in the Terraform Registry. The source address ryndaniels/xray I used above is the address this provider would take if the author were to publish it to the registry as-is, because the Terraform Registry uses a methodical naming scheme to generate provider source addresess from GitHub repository addresses.

But unless that provider is later published in the registry, you'll need to install it on your local system manually in order for Terraform to find it and use it. To achieve that, refer to Implied Local Mirror Directories to see which directories Terraform searches for plugins on your platform, and then you can create the necessary directory structure for Terraform to find the provider.

For the sake of an example I'm going to use the $HOME/.terraform.d/plugins/ prefix that Terraform supports on Linux, but note that the paths are different on Windows and macOS and so you will need to adjust to a different path prefix on those operating systems:

$HOME/.terraform.d/plugins/registry.terraform.io/ryndaniels/xray/0.0.1/linux_amd64/terraform-provider-xray

A few more notes about the above:

  • The registry.terraform.io/ryndaniels/xray part is what Terraform matches with the source you set in your provider_requirements block. registry.terraform.io is the default hostname for a source address that doesn't include a hostname, so ryndaniels/xray is short for registry.terraform.io/ryndaniels/xray.

  • This particular provider doesn't have any tagged releases and so there isn't a specific version number to use. Because Terraform expects all providers to have a version number, I arbitrarily chose 0.0.1 as the version number in the path above. This will only matter if you write an explicit version argument in your module to constrain which versions are allowed.

  • I used linux_amd64 as the target platform here, continuing my example of using Linux-style mirror paths. You can find out which platform is correct for you by running terraform version, which on recent Terraform versions includes the name of the platform that your Terraform CLI executable was built for:

    Terraform v0.14.4
on linux_amd64

On startup Terraform will scan all of the implied local mirror directories for your current platform and will note any providers it finds there, and assume that you intend to install those from your local filesystem rather than from the registry. Therefore as long as the directory structure is correct as above, terraform init will see that you have ryndaniels/xray installed locally and will use the copy from there rather than asking the Terraform Registry which versions are available for that provider.

If the provider author publishes this provider to the Terraform Registry at a later date, and they do so without renaming the repository or moving it into a different GitHub account, it should appear in the registry as ryndaniels/xray and thus terraform init will be able to automatically install it from there without any further modifications to your modules.

user_2818480
Related