# Sources

The `sources` key in `vendor.yaml` defines the list of components and artifacts to vendor. Each source specifies where to download from, what version, and where to place the files.

## Schema

```yaml
spec:
  sources:
    - component: "vpc"
      source: "github.com/org/repo.git//path?ref={{.Version}}"
      version: "1.0.0"
      targets:
        - "components/terraform/vpc"
      included_paths:
        - "**/*.tf"
      excluded_paths:
        - "**/test/**"
      tags:
        - networking
      identity: source-reader
      retry:
        max_attempts: 5
        initial_delay: 2s
        backoff_strategy: exponential
```

## Attributes

- **`component`** — The `component` attribute in each source is optional. It's used in the `atmos vendor pull --component <component>` command if the component is passed in. In this case, Atmos will vendor only the specified component instead of vendoring all the artifacts configured in the `vendor.yaml` manifest.
- **`version`** — The `version` attribute is used to specify the version of the artifact to download. The `version` attribute is used in the `source` and `targets` attributes as a template parameter using `{{ .Version }}`.
- **`source`** — The `source` attribute supports all protocols (local files, Git, Mercurial, HTTP, HTTPS, Amazon S3, Google GCP), and all the URL and archive formats as described in [go-getter](https://github.com/hashicorp/go-getter), and also the `oci://` scheme to download artifacts from [OCI registries](https://opencontainers.org).

  See [Vendor URL Syntax](/vendor/url-syntax) for complete documentation on supported URL formats, authentication, and subdirectory syntax.

  **IMPORTANT:** Include the `{{ .Version }}` parameter in your `source` URI to ensure the correct version of the artifact is downloaded.

  For example, for `http` and `https` sources, use the following format:
  ```yaml
  source: "github.com/cloudposse-terraform-components/aws-vpc-flow-logs-bucket.git?ref={{.Version}}"
  ```
  - **`ref`** — Pass the `ref` as a query string with either the tag, branch, or commit hash to download the correct version of the artifact. e.g. `?ref={{.Version}}` will pass the `version` attribute to the `ref` query string. Changing the resolved `ref` makes the next pull recognize the new source, even when local files match the previous lock receipt. With strict lock enforcement, pass `--refresh-lock` to accept the change.
  - **`depth`** — Pass the `depth` as a query string to download only the specified number of commits from the repository. e.g. `?depth=1` will download only the latest commit.
- **`identity`** — The name of an [Atmos Auth identity](/cli/configuration/auth/identities) used to fetch this source, for private repositories that need credentials the process does not already have. The identity's environment is applied only to the `git` commands that fetch this source, so sources vendored concurrently can use different identities.
  ```yaml
  spec:
    sources:
      - component: "vpc"
        source: "git::ssh://git@github.com/acme/modules.git//vpc?ref={{.Version}}"
        version: "1.4.0"
        identity: source-reader
        targets:
          - "components/terraform/vpc"
  ```
  The identity is chosen in this order: the `--identity` flag of [`atmos vendor pull`](/cli/commands/vendor/pull), then `identity` on the source, then [`vendor.identity`](/cli/configuration/vendor) in `atmos.yaml`. With none of these, fetching works as before, with the credentials already in your environment.

  Outside CI, an identity without a valid session fails with a hint to run `atmos auth login --identity <name>`. Atmos never prompts for a login in the middle of a fetch.

  A source that names an identity is fetched with that identity only. Atmos does not add a token from your environment (such as `GITHUB_TOKEN`) to its URL, and credentials that Atmos Pro provisioned for other fetches are not passed to it.

  Only Git sources use the identity's environment. S3 and OCI sources authenticate as before and do not need the identity's session. `atmos vendor update` and `atmos vendor diff` do not use the identity when they list available versions.
- **`targets`** — The `targets` in each source supports absolute paths and relative paths (relative to the `vendor.yaml` file). Note: if the `targets` paths
  are set as relative, and if the `vendor.yaml` file is detected by Atmos using the `base_path` setting in `atmos.yaml`, the `targets` paths
  will be considered relative to the `base_path`. Multiple targets can be specified.
- **`included_paths` and `excluded_paths`** — `included_paths` and `excluded_paths` support [POSIX-style greedy Globs](https://en.wikipedia.org/wiki/Glob_\(programming\)) for filenames/paths (double-star/globstar `**` is supported as well). For more details, see [Vendoring with Globs](#vendoring-with-globs) below.
- **`tags`** — The `tags` in each source specifies a list of tags to apply to the component. This allows you to only vendor the components that have the specified tags by executing a command `atmos vendor pull --tags <tag1>,<tag2>`
- **`retry`** — Optional retry configuration for handling transient network errors during download operations. This is useful for unreliable networks or when hitting rate limits.

  :::note
  When no retry configuration is provided, no retries are performed. All retry fields are optional—unspecified fields use zero values (e.g., `0` for attempts, `0s` for durations).
  :::
  ```yaml
  retry:
    max_attempts: 3
    initial_delay: 1s
    max_delay: 10s
    backoff_strategy: exponential
    multiplier: 2.0
    random_jitter: 0.1
    max_elapsed_time: 5m
  ```
  - **`max_attempts`** — Maximum number of retry attempts. Set to `1` or higher to enable retries.
  - **`initial_delay`** — Initial delay before the first retry (e.g., `1s`, `500ms`).
  - **`max_delay`** — Maximum delay between retries (e.g., `30s`, `1m`).
  - **`backoff_strategy`** — Strategy for increasing delay between retries. Values: `exponential`, `linear`, `constant`.
  - **`multiplier`** — Multiplier for exponential/linear backoff (e.g., `2.0`).
  - **`random_jitter`** — Random jitter factor (0.0-1.0) to add randomness to delays.
  - **`max_elapsed_time`** — Maximum total time for all retry attempts (e.g., `5m`, `1h`).

## Template Parameters

The `source` and `targets` attributes support [Go templates](https://pkg.go.dev/text/template) and [Sprig Functions](http://masterminds.github.io/sprig/).

- **`{{ .Component }}`** — Refers to the `component` attribute in the current section. The `component` attribute is used to specify the component name. This is useful to vendor components into folders by the same name.
  ```yaml
  targets:
    - "components/terraform/{{ .Component }}"
  ```
- **`{{ .Version }}`** — Refers to the `version` attribute the current section. The `version` attribute is used to specify the version of the artifact to download. This is useful to version components into different folders.
  ```yaml
  targets:
    - "components/terraform/{{ .Component }}/{{ .Version }}"
  ```
  When stacks need to pin to different versions of the same component, the `{{ .Version }}` template parameter can be used to ensure the components are vendored into different folders.

You can also use any of the [hundreds of go-template functions](/functions/template). For example, to extract the major and minor version from the `{{ .Version }}` attribute, use the following template:

```yaml
targets:
  - "components/terraform/{{ .Component }}/{{ (slice (splitList \".\" .Version) 0 2) | join \".\" }}"
```

## Authenticating to Private Git Repositories

Atmos provides **automatic token injection** for private repositories on GitHub, GitLab, and Bitbucket. This is the recommended approach for most users.

:::tip Automatic Token Injection (Recommended)
The easiest and most secure way to authenticate to private Git repositories is to use Atmos's automatic token injection.

**Step 1:** Set the authentication token as an environment variable:

```bash
export GITHUB_TOKEN="your-personal-access-token"
# or
export ATMOS_GITHUB_TOKEN="your-personal-access-token"
```

**Step 2:** Use simple URLs in your `vendor.yaml` (no manual credentials):

```yaml
sources:
  - component: "vpc"
    source: "github.com/your-org/private-repo.git//terraform/vpc?ref={{.Version}}"
    version: "1.0.0"
    targets:
      - "components/terraform/vpc"
```

**Step 3:** Atmos automatically injects the token when downloading.

Supported environment variables:

- **GitHub:** `ATMOS_GITHUB_TOKEN` or `GITHUB_TOKEN`
- **GitLab:** `ATMOS_GITLAB_TOKEN` or `GITLAB_TOKEN`
- **Bitbucket:** `ATMOS_BITBUCKET_TOKEN` or `BITBUCKET_TOKEN`

**Token injection settings in `atmos.yaml`:**

- `inject_github_token: true` (default: **true**) - Enables automatic GitHub token injection
- `inject_gitlab_token: true` (default: **false**) - Enables automatic GitLab token injection
- `inject_bitbucket_token: true` (default: **false**) - Enables automatic Bitbucket token injection
  :::

:::note GitHub Actions Token Scope
The `GITHUB_TOKEN` provided by GitHub Actions is only valid for the current repository, or repositories marked as `internal` within GitHub Enterprise organizations. For cross-repository access, provision a [fine grained personal access token](https://docs.github.com/en/rest/authentication/permissions-required-for-fine-grained-personal-access-tokens?apiVersion=2022-11-28) with the necessary permissions.
:::

## Vendoring from OCI Registries

Atmos supports vendoring from [OCI registries](https://opencontainers.org).

To specify a repository in an OCI registry, use the `oci://<registry>/<repository>:tag` scheme.

Artifacts from OCI repositories are downloaded as Docker image tarballs, then all the layers are processed, un-tarred and un-compressed,
and the files are written into the directories specified by the `targets` attribute of each `source`.

### OCI Authentication

Atmos uses the following precedence order for OCI registry authentication:

1. **Docker credentials** (highest precedence) - Credentials from `docker login` stored in `~/.docker/config.json`
2. **Environment variables** - For GitHub Container Registry (ghcr.io):
   - **Token:** `ATMOS_GITHUB_TOKEN` or `GITHUB_TOKEN`
   - **Username:** `ATMOS_GITHUB_USERNAME`, `GITHUB_ACTOR`, or `GITHUB_USERNAME`
3. **Anonymous** - Fallback for public images

### Example

**File:** `vendor.yaml`

```yaml
apiVersion: atmos/v1
kind: AtmosVendorConfig
metadata:
  name: example-vendor-config
  description: Atmos vendoring manifest
spec:
  sources:
    - component: "vpc"
      # Example image reference: replace with the OCI image containing the files you need.
      source: "oci://registry.example.com/team/{{ .Component }}:{{ .Version }}"
      version: "latest"
      targets:
        - "components/terraform/{{ .Component }}"
      included_paths:
        - "**/*.tf"
        - "**/*.tfvars"
        - "**/*.md"
      excluded_paths: []
```

To vendor the `vpc` component, execute the following command:

```shell
atmos vendor pull -c vpc
```

## Vendoring with Globs

In Atmos, **glob patterns** define which files and directories are included or excluded during vendoring. These patterns go beyond simple wildcard characters like `*`—they follow specific rules that dictate how paths are matched. Understanding the difference between **greedy** (`**`) and **non-greedy** (`*`) patterns, along with other advanced glob syntax, ensures precise control over vendoring behavior.

### Understanding Wildcards, Ranges, and Recursion

Glob patterns in Atmos provide flexible and powerful matching, that's simpler to understand than regular expressions:

- **`*` (single asterisk)** — Matches any sequence of characters within a single path segment.
- — — Example: `vendor/*.yaml` matches `vendor/config.yaml` but not `vendor/subdir/config.yaml`.
- **`**` (double asterisk, also known as a "greedy glob")** — Matches across multiple path segments recursively.
- — — Example: `vendor/**/*.yaml` matches `vendor/config.yaml`, `vendor/subdir/config.yaml`, and `vendor/deep/nested/config.yaml`.
- **`?` (question mark)** — Matches exactly one character in a path segment.
- — — Example: `file?.txt` matches `file1.txt` and `fileA.txt` but not `file10.txt`.
- **`[abc]` (character class)** — Matches any single character inside the brackets.
- — — Example: `file[123].txt` matches `file1.txt`, `file2.txt`, and `file3.txt`, but not `file4.txt` or `file12.txt`.
- **`[a-z]` (character range)** — Matches any single character within the specified range.
- — — Example: `file[a-c].txt` matches `filea.txt`, `fileb.txt`, and `filec.txt`.
- **`{a,b,c}` (brace expansion)** — Matches any of the comma-separated patterns.
- — — Example: `*.{jpg,png,gif}` matches `image.jpg`, `image.png`, and `image.gif`.

This distinction is important when excluding specific directories or files while vendoring.

### Example: Excluding a Subdirectory

Consider the following configuration:

```yaml
included_paths:
  - "**/demo-library/**"
excluded_paths:
  - "**/demo-library/**/stargazers/**"
```

How it works:

- The `included_paths` rule `**/demo-library/**` ensures all files inside `demo-library` (at any depth) are vendored.
- The `excluded_paths` rule `**/demo-library/**/stargazers/**` prevents any files inside `stargazers` subdirectories from being vendored.

This means:

- All files within `demo-library` except those inside any `stargazers` subdirectory are vendored.
- Any other files outside `stargazers` are unaffected by this exclusion.

### Example: Matching Multiple File Extensions

```yaml
included_paths:
  - "**/demo-library/**/*.{tf,md}"
```

This is equivalent to writing:

```yaml
included_paths:
  - "**/demo-library/**/*.tf"
  - "**/demo-library/**/*.md"
```

The `{tf,md}` part expands to both `*.tf` and `*.md`, making the rule more concise.

### Key Takeaways

1. Use `**/` for recursive matching to include everything inside a directory.
2. Use `*` for single-segment matches, which won't include deeper subdirectories.
3. Use `{...}` to match multiple extensions or directories within a single pattern.
4. Exclusion rules must match nested paths explicitly when trying to exclude deep directories.

## Try it out

### demo-vendoring

Copy the example into a new directory and make it your own:

```shell
atmos init examples/demo-vendoring
cd demo-vendoring
```

Inspect the configured sources before downloading components. Pulling sources requires network access.

```shell
atmos vendor list
```

- [Installation guide](/install)
- [Explore the example](/examples/demo-vendoring)
- [Initialization and repository settings](/cli/commands/init#examples-and-source-directories)
