Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ title: Changelog

## [Unreleased](https://github.com/lets-cli/lets/releases/tag/v0.0.X)

* `[Added]` Remote config support: `lets -c https://url` downloads and caches config to `~/.config/lets/remote-configs/`. Use `--no-cache` to force re-download. Only standalone configs (no `mixins:`) are supported for now.
* `[Added]` Add `lets self skills` commands to show, install, and update the bundled lets agent skill.
* `[Docs]` Document the bundled lets Agent Skill and link it from the config reference.
* `[Changed]` Expand the bundled lets agent skill with config authoring guidance.
Expand Down
87 changes: 87 additions & 0 deletions docs/specs/2026-06-13-remote-config-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
---
name: remote-config-design
description: Design for lets -c https://... remote config fetching with caching (issue #351)
---

# Remote Config Design

**Issue:** [#351](https://github.com/lets-cli/lets/issues/351)

## Summary

Allow `lets -c https://url` to download a remote `lets.yaml`, cache it locally, and run commands from it with the invocation directory as the working directory. Add `--no-cache` flag to force re-download.

## Architecture

### New flag: `--no-cache`

Added to `initRootFlags` in `internal/cmd/root.go` and parsed in `parseRootFlags` in `internal/cli/cli.go`, following the same pattern as `--debug` and `--init`.

### URL detection in `cli.go`

After parsing root flags, if `rootFlags.config` starts with `http://` or `https://`, call `config.LoadRemote(url, noCache, version)` instead of `config.Load(...)`. The existing `Load` path is untouched.

### New `internal/fetch` package

Extract HTTP download + content-type validation from `RemoteMixin.download()` into `internal/fetch/fetch.go`:

```go
func Download(ctx context.Context, url string) ([]byte, error)
```

Same 15-minute timeout and content-type whitelist (`text/plain`, `text/yaml`, `text/x-yaml`, `application/yaml`, `application/x-yaml`). `RemoteMixin` becomes a thin wrapper calling `fetch.Download`. This eliminates duplication.

### New `LoadRemote` in `internal/config/load.go`

```go
func LoadRemote(url string, noCache bool, version string) (*config.Config, error)
```

- Cache path: `util.LetsUserDir()` + `remote-configs/<hex(sha256(url))>.yaml`
- Resolves to `~/.config/lets/remote-configs/<sha256>.yaml`
- Creates `~/.config/lets/remote-configs/` if needed
- Working directory: `os.Getwd()` (CWD at invocation time), passed as `configDir` to `Load`

## Data Flow

### Normal flow (no `--no-cache`)

1. Cache file exists → load directly from cached path, skip HTTP
2. Cache missing → download via `fetch.Download` → persist → load from cached path

### `--no-cache` flow

1. Attempt download → persist (overwriting cache) → load
2. Download fails + cache exists → `log.Warnf("failed to refresh remote config, using cached version: %s", err)` → load from cache
3. Download fails + no cache → return error

### Working directory

`LoadRemote` calls `Load(cachedPath, cwd, version)` where `cwd = os.Getwd()`. Commands in the remote config execute in the invocation directory unless they specify `work_dir`.

## Files Changed

| File | Change |
|------|--------|
| `internal/fetch/fetch.go` | New — extracted `Download` func |
| `internal/fetch/fetch_test.go` | New — unit tests for fetch |
| `internal/config/config/mixin.go` | Refactor `download()` to use `fetch.Download` |
| `internal/config/load.go` | Add `LoadRemote` |
| `internal/config/load_test.go` | Add `LoadRemote` tests |
| `internal/cli/cli.go` | URL detection, call `LoadRemote`, thread `noCache` |
| `internal/cmd/root.go` | Add `--no-cache` flag |

## Testing

### `internal/fetch/fetch_test.go`
- Valid YAML content-type → success
- Unsupported content-type → error
- 404 → error
- Non-2xx → error

### `internal/config/load_test.go`
- `LoadRemote` with `httptest.NewServer` serving valid YAML → loads, caches, runs from CWD
- Cache hit: serve once, call twice, assert server receives only one request
- `--no-cache` refresh: serve two different configs, assert second call gets new content
- `--no-cache` + server down + cache exists: assert warning logged + falls back to cache
- `--no-cache` + server down + no cache: assert error returned
25 changes: 22 additions & 3 deletions internal/cli/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ import (

"github.com/lets-cli/fang"
"github.com/lets-cli/lets/internal/cmd"
"github.com/lets-cli/lets/internal/config"
"github.com/lets-cli/lets/internal/config/config"
loader "github.com/lets-cli/lets/internal/config"
"github.com/lets-cli/lets/internal/env"
"github.com/lets-cli/lets/internal/logging"
"github.com/lets-cli/lets/internal/set"
Expand Down Expand Up @@ -77,7 +78,16 @@ func Main(version string, buildDate string) int {
rootFlags.config = os.Getenv("LETS_CONFIG")
}

cfg, err := config.Load(rootFlags.config, configDir, version)
var cfg *config.Config
if isRemoteURL(rootFlags.config) {
if configDir != "" {
log.Warnf("LETS_CONFIG_DIR is ignored when using a remote config URL")
}

cfg, err = loader.LoadRemote(ctx, rootFlags.config, rootFlags.noCache, version)
} else {
cfg, err = loader.Load(rootFlags.config, configDir, version)
}
if err != nil {
if failOnConfigError(rootCmd, command, rootFlags) {
log.Errorf("config error: %s", err)
Expand Down Expand Up @@ -270,13 +280,18 @@ func isInteractiveStderr() bool {
return isatty.IsTerminal(fd) || isatty.IsCygwinTerminal(fd)
}

func isRemoteURL(s string) bool {
return strings.HasPrefix(s, "http://") || strings.HasPrefix(s, "https://")
}

type flags struct {
config string
debug int
help bool
version bool
all bool
init bool
noCache bool
}

// We can not parse --config and --debug flags using cobra.Command.ParseFlags
Expand Down Expand Up @@ -326,7 +341,7 @@ func parseRootFlags(args []string) (*flags, error) {
}

f.config = value
} else if len(args[idx:]) > 0 {
} else if idx+1 < len(args) {
f.config = args[idx+1]
idx += 2

Expand Down Expand Up @@ -356,6 +371,10 @@ func parseRootFlags(args []string) (*flags, error) {
if !isFlagVisited("init") {
f.init = true
}
case "--no-cache":
if !isFlagVisited("no-cache") {
f.noCache = true
}
case "--upgrade":
return nil, errors.New("--upgrade has been replaced with 'lets self upgrade'")
}
Expand Down
1 change: 1 addition & 0 deletions internal/cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -108,4 +108,5 @@ func initRootFlags(rootCmd *cobra.Command) {
rootCmd.Flags().CountP("debug", "d", "show debug logs (or use LETS_DEBUG=1). If used multiple times, shows more verbose logs") //nolint:lll
rootCmd.Flags().StringP("config", "c", "", "config file (default is lets.yaml)")
rootCmd.Flags().Bool("all", false, "show all commands (including the ones with _)")
rootCmd.Flags().Bool("no-cache", false, "re-download remote config instead of using cached version")
}
1 change: 1 addition & 0 deletions internal/cmd/testdata/TestHelpGolden/basic.golden
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ FLAGS
--exclude Run all but excluded command(s) described in cmd as map
-h --help Help for lets
--init Create a new lets.yaml in the current folder
--no-cache Re-Download remote config instead of using cached version
--no-depends Skip 'depends' for running command
--only Run only specified command(s) described in cmd as map
-v --version Version for lets
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ FLAGS
--exclude Run all but excluded command(s) described in cmd as map
-h --help Help for lets
--init Create a new lets.yaml in the current folder
--no-cache Re-Download remote config instead of using cached version
--no-depends Skip 'depends' for running command
--only Run only specified command(s) described in cmd as map
-v --version Version for lets
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ FLAGS
--exclude Run all but excluded command(s) described in cmd as map
-h --help Help for lets
--init Create a new lets.yaml in the current folder
--no-cache Re-Download remote config instead of using cached version
--no-depends Skip 'depends' for running command
--only Run only specified command(s) described in cmd as map
-v --version Version for lets
Expand Down
1 change: 1 addition & 0 deletions internal/cmd/testdata/TestHelpGolden/long_command.golden
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ FLAGS
--exclude Run all but excluded command(s) described in cmd as map
-h --help Help for lets
--init Create a new lets.yaml in the current folder
--no-cache Re-Download remote config instead of using cached version
--no-depends Skip 'depends' for running command
--only Run only specified command(s) described in cmd as map
-v --version Version for lets
Expand Down
3 changes: 3 additions & 0 deletions internal/config/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@ type Config struct {
// absolute path to .lets/mixins
MixinsDir string

// RemoteSource is the original URL when the config was loaded remotely; empty for local configs.
RemoteSource string

// cached env after config.SetupEnv, used in config.GetEnv
cachedEnv map[string]string
isMixin bool // if true, we consider config as mixin and apply different parsing and validation
Expand Down
71 changes: 2 additions & 69 deletions internal/config/config/mixin.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,38 +5,14 @@ import (
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"mime"
"net/http"
"os"
"path/filepath"
"strings"
"time"

"github.com/lets-cli/lets/internal/set"
"github.com/lets-cli/lets/internal/fetch"
"github.com/lets-cli/lets/internal/util"
)

var allowedContentTypes = set.NewSet(
"text/plain",
"text/yaml",
"text/x-yaml",
"application/yaml",
"application/x-yaml",
)

// normalizeContentType extracts the media type from a Content-Type header,
// removing parameters like charset to enable robust matching.
func normalizeContentType(contentType string) string {
mediaType, _, err := mime.ParseMediaType(contentType)
if err != nil {
// If parsing fails, return the original string
return contentType
}

return mediaType
}

type Mixins []*Mixin

type Mixin struct {
Expand Down Expand Up @@ -102,50 +78,7 @@ func (rm *RemoteMixin) tryRead() ([]byte, error) {
}

func (rm *RemoteMixin) download() ([]byte, error) {
// TODO: maybe create a client for this?
ctx, cancel := context.WithTimeout(context.Background(), 60*5*time.Second)
defer cancel()

req, err := http.NewRequestWithContext(
ctx,
http.MethodGet,
rm.URL,
nil,
)
if err != nil {
return nil, err
}

client := &http.Client{
Timeout: 15 * 60 * time.Second, // TODO: move to client struct
}

resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("failed to make request: %w", err)
}

defer resp.Body.Close()

if resp.StatusCode == http.StatusNotFound {
return nil, fmt.Errorf("no such file at: %s", rm.URL)
} else if resp.StatusCode < 200 || resp.StatusCode > 299 {
return nil, fmt.Errorf("network error: %s", resp.Status)
}

contentType := resp.Header.Get("Content-Type")

normalizedContentType := normalizeContentType(contentType)
if !allowedContentTypes.Contains(normalizedContentType) {
return nil, fmt.Errorf("unsupported content type: %s", contentType)
}

data, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("failed to read response: %w", err)
}

return data, nil
return fetch.Download(context.Background(), rm.URL)
}

// Trim `-` prefix.
Expand Down
Loading
Loading