Support loading dataset by tag

[dentiny]

Closes #83

Hi team, in this PR I add multi-version dataset viewing. The viewer continues to load the latest snapshot from main by default, while allowing users to select a numeric version or tag for ML experimentation and dataset comparison.

Implementation-wise, a few changes

• Add a dataset-location field when DATA_PATH is not configured.
• Add an optional reference field supporting:
  • main for the latest snapshot
  • A numeric version such as 1
  • A tag such as tag:base-snapshot
• Propagate the selected location and reference through dataset API requests.
• Display reference and dataset-loading errors in the UI.

Landing page

Loading dataset by default "main"

Loading dataset by tag and version

[gordonmurray]

Thanks Destiny, This looks promising, and the existing test suite is passing.

Could you rebase onto the current main and rerun CI? The branch predates the current 0.36.0 canonical image, so the latest compatibility path still needs verification.

After that, I think this should be ready for approval pending the manual version/tag checks.

[dentiny]

Could you rebase onto the current main and rerun CI?

For sure, done.

[gordonmurray]

Thanks for the rebase. The version and tag selection works well. I built a table with two versions and a tag, and main, 1, and tag:base-snapshot each returned the right rows, with total matching the selected version rather than the latest one.

I pushed two commits to your branch rather than send you round again. Both are small. If you disagree with either, say so and I will drop it.

The first restores ENV DATA_PATH=/data in the Dockerfile. Without it the run command in the README stops working, so anyone who upgrades to :latest gets an empty location box in place of their data:

docker run -p 8080:8080 -v $PWD/data:/data:ro <image>

main:        GET /datasets -> 200 {"datasets":["sample"]}
this branch: GET /datasets -> 400 "A Lance dataset location is required when DATA_PATH is not set"

Your location mode is still there. It turns on with -e DATA_PATH=, and I updated the README to describe that. I also reverted the -e DATA_PATH=/data line added to the smoke test, because that check exists to catch this kind of break and the flag hid it. With DATA_PATH unset the startup checks in entrypoint.sh were skipped as well, so the default brings those back.

The second adds a guard before lancedb.connect(). For a local path, connect() runs Path(uri).mkdir(parents=True, exist_ok=True), so a location that does not exist gets created on the server. A missing local location now returns 400 and writes nothing. Remote URIs skip the check, because connect() creates nothing for s3://.

All eight versions in the matrix pass, 42 tests each.

[dentiny]

Sounds good, thanks for the change!