19 Commits
v0.4.0 ... main

Author SHA1 Message Date
aa1602efde docs: clean up CHANGELOG.md — remove 40 duplicate section blocks and RU mirror (#122) 2026-04-12 00:56:09 +03:00
b3239ed2d7 chore: DX improvements — typos, blame-ignore, PR template, registry checklist (#120)
* chore: add workspace clippy lints, release profiles, COMPAT.md, diff-registry.sh

- Workspace clippy lints: or_fun_call, redundant_clone, collection_is_never_read,
  naive_bytecount, stable_sort_primitive, large_types_passed_by_value, assigning_clones
- Fix or_fun_call in cargo_registry.rs (unwrap_or -> unwrap_or_else)
- Release profiles: release (thin LTO) + release-official (full LTO, codegen-units=1)
- COMPAT.md: protocol compatibility matrix for all 7 registries (40 endpoints)
- scripts/diff-registry.sh: differential smoke tests (Docker/npm/Cargo/PyPI/Go/Raw)

* ci: add typos spell-check job and config

* chore: add .git-blame-ignore-revs for bulk fmt/clippy commits

* chore: unify PR template with What/Why/Checklist format

* docs: add new registry checklist and improve contributing guide

* fix: correct typos action SHA to v1.45.0
2026-04-09 18:49:20 +03:00
e4168b7ee4 chore: add workspace clippy lints, release profiles, COMPAT.md, diff-registry.sh (#119)
- Workspace clippy lints: or_fun_call, redundant_clone, collection_is_never_read,
  naive_bytecount, stable_sort_primitive, large_types_passed_by_value, assigning_clones
- Fix or_fun_call in cargo_registry.rs (unwrap_or -> unwrap_or_else)
- Release profiles: release (thin LTO) + release-official (full LTO, codegen-units=1)
- COMPAT.md: protocol compatibility matrix for all 7 registries (40 endpoints)
- scripts/diff-registry.sh: differential smoke tests (Docker/npm/Cargo/PyPI/Go/Raw)
2026-04-09 12:38:59 +03:00
fda562fe21 chore: higher resolution dashboard screenshot (1920x1080) (#118) 2026-04-08 12:13:11 +03:00
e79b0f58f7 chore: update dashboard screenshot to v0.5.0 with all 7 registries (#117) 2026-04-08 12:08:03 +03:00
388ea8f6a5 test: add Go and Raw to Playwright dashboard and health checks (#116)
All 7 registry types now verified in UI visibility and health endpoint tests.
2026-04-08 11:56:51 +03:00
4003c54744 fix: add missing Go and Raw registries to OpenAPI, startup logs, and metrics tests (#115)
- openapi.rs: add Go Module Proxy and Raw File Storage tags, update version and description
- main.rs: add /go/ to Available endpoints startup log
- dashboard_metrics.rs: add go to download and upload test coverage for all 7 registries
2026-04-08 11:39:36 +03:00
71d8d83585 docs: add community contributions to v0.5.0 changelog (@TickTockBent PRs #97, #108, #109) (#114) 2026-04-08 09:53:39 +03:00
27a368b3a0 feat: Cargo sparse index (RFC 2789) + PyPI twine upload + PEP 691 (#113)
Cargo registry:
- Sparse index with config.json, prefix-based lookup (1/2/3/4+ char rules)
- cargo publish wire format (LE u32 lengths + JSON metadata + tarball)
- Version immutability with Cargo-compatible JSON error responses
- Dependency field mapping (version_req->req, explicit_name_in_toml->package)
- Case-insensitive crate name normalization across all endpoints
- Cache-Control headers on index (max-age=300) and downloads (immutable)

PyPI registry:
- twine upload via multipart/form-data with SHA-256 verification
- PEP 691 JSON API with Accept header content negotiation
- Hash fragment preservation in proxied links (PEP 503)
- Package name normalization per PEP 503

577 tests (up from 504), 0 failures, clippy clean.
2026-04-08 09:38:18 +03:00
dependabot[bot]
25ba9f6cb5 chore(deps): bump criterion from 0.5.1 to 0.8.2 (#112)
Bumps [criterion](https://github.com/criterion-rs/criterion.rs) from 0.5.1 to 0.8.2.
- [Release notes](https://github.com/criterion-rs/criterion.rs/releases)
- [Changelog](https://github.com/criterion-rs/criterion.rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/criterion-rs/criterion.rs/compare/0.5.1...criterion-v0.8.2)

---
updated-dependencies:
- dependency-name: criterion
  dependency-version: 0.8.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: DevITWay | Pavel Volkov <devitway@gmail.com>
2026-04-07 17:02:28 +03:00
dependabot[bot]
347e3016d3 chore(deps): bump toml from 1.0.6+spec-1.1.0 to 1.1.2+spec-1.1.0 (#111)
Bumps [toml](https://github.com/toml-rs/toml) from 1.0.6+spec-1.1.0 to 1.1.2+spec-1.1.0.
- [Commits](https://github.com/toml-rs/toml/compare/toml-v1.0.6...toml-v1.1.2)

---
updated-dependencies:
- dependency-name: toml
  dependency-version: 1.1.2+spec-1.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: DevITWay | Pavel Volkov <devitway@gmail.com>
2026-04-07 16:59:42 +03:00
dependabot[bot]
7367670114 chore(deps): bump docker/login-action from 4.0.0 to 4.1.0 (#110)
Bumps [docker/login-action](https://github.com/docker/login-action) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](b45d80f862...4907a6ddec)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-07 16:56:46 +03:00
Wes
9d3c248ea5 fix: scope garbage collection to Docker-only blobs (#109)
collect_all_blobs scanned all seven registry prefixes for keys
containing /blobs/ or /tarballs/, but collect_referenced_digests only
reads Docker manifests. Non-Docker artifacts (notably npm tarballs)
were collected but never marked as referenced, so running gc without
--dry-run would delete them.

Restrict blob collection to docker/ until per-registry reference
resolution exists for other registry types.

// ticktockbent
2026-04-06 20:28:21 +03:00
Wes
98c9c33ea0 fix: correct zeroize annotation placement and avoid secret cloning in protected.rs (#108)
S3Credentials had #[zeroize(skip)] on secret_access_key instead of
access_key_id. The comment said "access_key_id is not sensitive" but
the annotation was on the adjacent field. In practice ProtectedString
has its own #[zeroize(drop)] so the secret was still zeroed on drop,
but the annotation contradicted its own comment and would silently
break if ProtectedString's drop behavior ever changed.

ProtectedString::into_inner() cloned self.inner instead of moving it
out. Use std::mem::take to swap the value out directly, avoiding the
unnecessary heap allocation of the clone.

// ticktockbent
2026-04-06 15:30:18 +03:00
d7deae9b30 fix: cargo upstream proxy + mirror UX improvements (#107)
- Add CargoConfig with upstream proxy to crates.io (NORA_CARGO_PROXY)
- Cargo download/metadata: try local storage first, fetch from upstream on miss, cache in background
- Detect yarn.lock in npm subcommand and suggest correct command
- Add pip transitive dependency warning for air-gapped installs
- Improve mirror error messages with HTTP status codes
2026-04-06 13:33:13 +03:00
38828ec31e chore: SLSA L3 provenance, cosign .sig, Artifact Hub (#106)
- Add SLSA L3 provenance via slsa-github-generator reusable workflow
- Build job outputs binary hash for provenance generation
- Cosign now outputs .sig + .cert alongside .bundle (scorecard needs .sig)
- Remove fake provenance JSON and broken gh attestation step
- Add artifacthub-repo.yml for Artifact Hub discovery
2026-04-06 02:53:22 +03:00
0c95fa9786 fix: revert codeql-action to tag in scorecard.yml (#105)
Scorecard webapp verifies upload-sarif action by tag, not SHA.
Pinning to SHA causes imposter commit error on webapp submission.
2026-04-06 02:38:06 +03:00
69b7f1fb4e chore: pin dependencies to SHA digests for OpenSSF scorecard (#104)
Pin scorecard-action and codeql-action to commit SHA in scorecard.yml.
Pin base images to digest in Dockerfile.redos and Dockerfile.astra.
Replace curl|bash with direct binary download for actionlint.
Remove unused pip install cargo-audit-sarif.
2026-04-06 02:28:02 +03:00
Wes
b949ef49b8 fix: add Go and Raw registries to metrics detection and health endpoint (#97)
detect_registry() in metrics.rs was missing branches for /go/ and /raw/
paths, causing all requests to those registries to be labeled "other" in
Prometheus metrics. dashboard_metrics.rs already lists all seven
registries, so this was an oversight.

Also adds go and raw fields to RegistriesHealth so the /health endpoint
reports all seven registries consistently.

Fixes the test_detect_registry_go_path test which previously asserted
the wrong behavior ("other" instead of "go"), and adds tests for raw
path detection and prefix collision safety.

// ticktockbent

Co-authored-by: DevITWay | Pavel Volkov <devitway@gmail.com>
2026-04-05 23:09:23 +00:00
33 changed files with 2514 additions and 914 deletions

32
.git-blame-ignore-revs Normal file
View File

@@ -0,0 +1,32 @@
# Bulk formatting and lint-fix commits — ignore in git blame
# See: https://docs.github.com/en/repositories/working-with-files/using-files/viewing-a-file#ignore-commits-in-the-blame-view
# style: cargo fmt
b2be7102fef2b42d7546ef66c8fd02f06b130bc4
26d30b622dc4d17d160999f043e8be9985cea263
8da4c4278a87ced1420fc6e7a9ea2c567e4e7b97
# style: apply rustfmt to registry handlers
8336166e0e541f213d0f3b20d55ea509bbb2f2d8
# style: fix formatting
a9125e6287e9f31fff0720e7c1c07cdc5e94c9db
bbdefff07cf588ad5f848bec9031f4e51cc47c41
ac4020d34f72b08e1eb3dc0c4248128b1012ddb5
# Fix formatting
08eea07cfe05ac64e9d6d4a7f8314f269d834e9c
c7098a4aed2a880dff418abe48c5016ea5ac20e0
# Fix code formatting
0a97b00278c59a267c0fc7cdca7eb2bd7aa5decf
# Fix clippy warnings
cf9feee5b2116e216cbcd6b0d3ae1fe5e93cf7d5
2f86b4852a9c9a1a5691e8b48da8be3fb45f6d0c
# fix: clippy let_and_return warning
dab3ee805edbd2e6fb3cffda9c9618468880153e
# fix: resolve clippy warnings and format code
00fbd201127defee9c24a8edeb01eba3c053f306

View File

@@ -1,16 +1,14 @@
## Summary
## What
<!-- What does this PR do? -->
<!-- Brief description of changes -->
## Changes
## Why
<!-- List key changes -->
<!-- Motivation / issue reference -->
## Checklist
- [ ] passes
- [ ] passes
- [ ] passes
- [ ] No in production code
- [ ] New public API has documentation
- [ ] CHANGELOG updated (if user-facing change)
- [ ] Tests pass (`cargo test`)
- [ ] No new clippy warnings (`cargo clippy -- -D warnings`)
- [ ] Updated CHANGELOG.md (if user-facing change)
- [ ] New registry? See CONTRIBUTING.md checklist

Binary file not shown.

Before

Width:  |  Height:  |  Size: 124 KiB

BIN
.github/assets/dashboard.png vendored Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 279 KiB

View File

@@ -1,15 +0,0 @@
## What does this PR do?
<!-- Brief description of the change -->
## Related issue
<!-- Link to issue, e.g. Fixes #123 -->
## Checklist
- [ ] `cargo fmt` passes
- [ ] `cargo clippy` passes with no warnings
- [ ] `cargo test --lib --bin nora` passes
- [ ] New functionality includes tests
- [ ] CHANGELOG.md updated (if user-facing change)

View File

@@ -9,6 +9,13 @@ on:
permissions: read-all
jobs:
typos:
name: Typos
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: crate-ci/typos@02ea592e44b3a53c302f697cddca7641cd051c3d # v1.45.0
test:
name: Test
runs-on: ubuntu-latest
@@ -37,7 +44,10 @@ jobs:
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Install actionlint
run: bash <(curl -s https://raw.githubusercontent.com/rhysd/actionlint/main/scripts/download-actionlint.bash)
run: |
ACTIONLINT_VERSION=1.7.12
curl -sLO "https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz"
tar xzf "actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" actionlint
- name: Run actionlint
run: ./actionlint -ignore "shellcheck reported issue" -ignore "SC[0-9]"
@@ -116,7 +126,6 @@ jobs:
- name: Upload cargo-audit results as SARIF
if: always()
run: |
pip install --quiet cargo-audit-sarif 2>/dev/null || true
python3 -c "
import json, sys
sarif = {

View File

@@ -15,6 +15,8 @@ jobs:
build:
name: Build & Push
runs-on: [self-hosted, nora]
outputs:
hash: ${{ steps.hash.outputs.hash }}
permissions:
contents: read
packages: write
@@ -41,13 +43,20 @@ jobs:
path: ./nora
retention-days: 1
- name: Compute binary hash for SLSA provenance
id: hash
run: |
cp target/x86_64-unknown-linux-musl/release/nora ./nora-linux-amd64
sha256sum nora-linux-amd64 | base64 -w0 > hash.txt
echo "hash=$(cat hash.txt)" >> $GITHUB_OUTPUT
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4
with:
driver-opts: network=host
- name: Log in to GitHub Container Registry
uses: docker/login-action@b45d80f862d83dbcd57f89517bcf500b2ab88fb2 # v4
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -185,10 +194,22 @@ jobs:
sarif_file: trivy-image-${{ matrix.name }}.sarif
category: trivy-image-${{ matrix.name }}
provenance:
name: SLSA Provenance
needs: build
permissions:
actions: read
id-token: write
contents: write
uses: slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@v2.1.0
with:
base64-subjects: "${{ needs.build.outputs.hash }}"
upload-assets: true
release:
name: GitHub Release
runs-on: [self-hosted, nora]
needs: [build, scan]
needs: [build, scan, provenance]
permissions:
contents: write
id-token: write # Sigstore cosign keyless signing
@@ -215,42 +236,6 @@ jobs:
echo "Binary size: $(du -sh nora-linux-amd64 | cut -f1)"
cat nora-linux-amd64.sha256
- name: Generate SLSA provenance
uses: slsa-framework/slsa-github-generator/.github/actions/generate-builder@f7dd8c54c2067bafc12ca7a55595d5ee9b75204a # v2.1.0
id: provenance-generate
continue-on-error: true
- name: Upload provenance attestation
if: always()
run: |
# Generate provenance using gh attestation (built-in GitHub feature)
gh attestation create ./nora-linux-amd64 --repo ${{ github.repository }} --signer-workflow ${{ github.server_url }}/${{ github.repository }}/.github/workflows/release.yml 2>/dev/null || echo "WARNING: attestation failed, continuing without provenance"
# Also create a simple provenance file for scorecard
cat > nora-v${{ github.ref_name }}.provenance.json << 'PROVEOF'
{
"_type": "https://in-toto.io/Statement/v0.1",
"predicateType": "https://slsa.dev/provenance/v0.2",
"subject": [{"name": "nora-linux-amd64"}],
"predicate": {
"builder": {"id": "${{ github.server_url }}/${{ github.repository }}/.github/workflows/release.yml"},
"buildType": "https://github.com/slsa-framework/slsa-github-generator/generic@v2",
"invocation": {
"configSource": {
"uri": "${{ github.server_url }}/${{ github.repository }}",
"digest": {"sha1": "${{ github.sha }}"},
"entryPoint": ".github/workflows/release.yml"
}
},
"metadata": {
"buildInvocationID": "${{ github.run_id }}",
"completeness": {"parameters": true, "environment": false, "materials": false}
}
}
}
PROVEOF
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Generate SBOM (SPDX)
uses: anchore/sbom-action@57aae528053a48a3f6235f2d9461b05fbcb7366d # v0
with:
@@ -267,7 +252,12 @@ jobs:
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v3
- name: Sign binary with cosign (keyless Sigstore)
run: cosign sign-blob --yes --bundle nora-linux-amd64.bundle ./nora-linux-amd64
run: |
cosign sign-blob --yes \
--output-signature nora-linux-amd64.sig \
--output-certificate nora-linux-amd64.cert \
--bundle nora-linux-amd64.bundle \
./nora-linux-amd64
- name: Create Release
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2
@@ -276,10 +266,11 @@ jobs:
files: |
nora-linux-amd64
nora-linux-amd64.sha256
nora-linux-amd64.sig
nora-linux-amd64.cert
nora-linux-amd64.bundle
nora-${{ github.ref_name }}.sbom.spdx.json
nora-${{ github.ref_name }}.sbom.cdx.json
nora-${{ github.ref_name }}.provenance.json
body: |
## Install

View File

@@ -24,7 +24,7 @@ jobs:
persist-credentials: false
- name: Run OpenSSF Scorecard
uses: ossf/scorecard-action@v2.4.3
uses: ossf/scorecard-action@4eaacf0543bb3f2c246792bd56e8cdeffafb205a # v2.4.3
with:
results_file: results.sarif
results_format: sarif

View File

@@ -1,4 +1,33 @@
# Changelog
## [Unreleased]
## [0.5.0] - 2026-04-07
### Added
- **Cargo sparse index (RFC 2789)** — cargo can now use NORA as a proper registry with `sparse+http://` protocol, including `config.json`, prefix-based index lookup, and `cargo publish` wire format support
- **Cargo publish** — full publish flow with wire format parsing, version immutability (409 Conflict), SHA-256 checksums in sparse index, and proper `warnings` response format
- **PyPI twine upload** — `twine upload` via multipart/form-data with SHA-256 verification, filename validation, and version immutability
- **PEP 691 JSON API** — content negotiation via `Accept: application/vnd.pypi.simple.v1+json` for package index and version listing, with hash digests in responses
- 577 total tests (up from 504), including 25 new Cargo tests and 18 new PyPI tests
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Cargo dependency field mapping: `version_req` correctly renamed to `req` and `explicit_name_in_toml` to `package` in sparse index entries, matching Cargo registry specification
- Cargo crate names normalized to lowercase across all endpoints (publish, download, metadata, sparse index) for consistent storage keys
- Cargo publish write ordering: index written before .crate tarball to prevent orphaned files on partial failure
- Cargo conflict errors now return Cargo-compatible JSON format (`{"errors": [{"detail": "..."}]}`)
- PyPI hash fragments preserved when rewriting upstream links (PEP 503 compliance)
- Redundant path traversal checks removed from crate name validation (charset already excludes unsafe characters)
### Changed
- Cargo sparse index and config.json responses include `Cache-Control: public, max-age=300`
- Cargo .crate downloads include `Cache-Control: public, max-age=31536000, immutable` and `Content-Type: application/x-tar`
- axum upgraded with `multipart` feature for PyPI upload support
## [0.4.0] - 2026-04-05
### Added
@@ -12,6 +41,10 @@
- fetch_blob_from_upstream and fetch_manifest_from_upstream are now pub for reuse in mirror module
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- tarpaulin exclude-files paths corrected to workspace-relative (coverage jumped from 29% to 61%) (#92)
- Env var naming unified across all registries (#39, #90)
@@ -30,6 +63,10 @@
- clippy.toml added for consistent lint rules
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Proxy request deduplication — concurrent requests coalesced (#83)
- Multi-registry GC now handles all 7 registry types (#83)
- TOCTOU race condition in credential validation (#83)
@@ -66,6 +103,10 @@
- README restructured: roadmap in README, removed stale ROADMAP.md (#65, #66)
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Remove all unwrap() from production code — proper error handling throughout (#72)
- Add `#![forbid(unsafe_code)]` — no unsafe code allowed at crate level (#72)
- Add input validation to Cargo registry endpoints (#72)
@@ -86,6 +127,10 @@
- **Anonymous read mode** (`NORA_AUTH_ANONYMOUS_READ=true`): allow pull/download without credentials while requiring auth for push. Use case: public demo registries, read-only mirrors.
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Pin slsa-github-generator and codeql-action by SHA instead of tag
- Replace anonymous tuple with named struct in activity grouping (readability)
- Replace unwrap() with if-let pattern in activity grouping (safety)
@@ -94,6 +139,10 @@
## [0.2.34] - 2026-03-20
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- **UI**: Group consecutive identical activity entries — repeated cache hits show as "artifact (x4)" instead of 4 identical rows
- **UI**: Fix table cell padding in Mount Points and Activity tables — th/td alignment now consistent
- **Security**: Update tar crate 0.4.44 → 0.4.45 (CVE-2026-33055 PAX size header bypass, CVE-2026-33056 symlink chmod traversal)
@@ -120,6 +169,10 @@
- Run containers as non-root user (USER nora) in all Dockerfiles
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Filter .meta.json from Docker tag list (fixes ArgoCD Image Updater tag recursion)
- Fix catalog endpoint to show namespaced images correctly (library/alpine instead of library)
@@ -141,7 +194,6 @@
- **CI**: Исправлены проверки лицензий cargo-deny
## [0.2.31] - 2026-03-16
### Added / Добавлено
@@ -169,9 +221,6 @@
- **npm proxy_auth**: Поле `proxy_auth` было в конфиге, но не передавалось в `fetch_from_proxy` — теперь отправляет Basic Auth в upstream
All notable changes to NORA will be documented in this file.
---
## [0.2.30] - 2026-03-16
@@ -203,19 +252,6 @@ All notable changes to NORA will be documented in this file.
### Removed / Удалено
- Removed unused `DockerAuth::fetch_with_auth()` method (dead code cleanup)
- Удалён неиспользуемый метод `DockerAuth::fetch_with_auth()` (очистка мёртвого кода)
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.28] - 2026-03-13
### Fixed / Исправлено
@@ -233,19 +269,6 @@ All notable changes to NORA will be documented in this file.
### Removed / Удалено
- Removed stale `CHANGELOG.md.bak` from repository
- Удалён устаревший `CHANGELOG.md.bak` из репозитория
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.27] - 2026-03-03
### Added / Добавлено
@@ -259,19 +282,6 @@ All notable changes to NORA will be documented in this file.
### Fixed / Исправлено
- Docker push of images >100MB no longer fails with 413 error
- Push Docker-образов >100MB больше не падает с ошибкой 413
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.26] - 2026-03-03
### Added / Добавлено
@@ -293,19 +303,6 @@ All notable changes to NORA will be documented in this file.
### Security / Безопасность
- Read-only tokens (`role: read`) are now blocked from PUT/POST/DELETE/PATCH operations with HTTP 403
- Токены только для чтения (`role: read`) теперь блокируются при PUT/POST/DELETE/PATCH с HTTP 403
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.25] - 2026-03-03
### Fixed / Исправлено
@@ -329,19 +326,6 @@ All notable changes to NORA will be documented in this file.
- `docker/build-push-action` 5 → 6
- Move scan/release to self-hosted runner with NORA cache
- Сканирование/релиз перенесены на self-hosted runner с кэшем через NORA
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.24] - 2026-02-24
### Added / Добавлено
@@ -351,19 +335,6 @@ All notable changes to NORA will be documented in this file.
### CI/CD
- Restore Astra Linux SE Docker image build, Trivy scan, and release artifact (`-astra` tag)
- Восстановлена сборка Docker-образа для Astra Linux SE, сканирование Trivy и артефакт релиза (тег `-astra`)
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.23] - 2026-02-24
### Added / Добавлено
@@ -396,37 +367,11 @@ All notable changes to NORA will be documented in this file.
### Documentation / Документация
- Replace text title with SVG logo; `O` styled in blue-600 / Заголовок заменён SVG-логотипом; буква `O` стилизована в blue-600
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.22] - 2026-02-24
### Changed / Изменено
- First stable release with Docker images published to container registry
- Первый стабильный релиз с Docker-образами, опубликованными в container registry
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.21] - 2026-02-24
### CI/CD
@@ -438,19 +383,6 @@ All notable changes to NORA will be documented in this file.
- Use GitHub-runner's own Rust toolchain (avoid path conflicts) / Используется Rust toolchain самого GitHub-runner'а
- Use shared runner filesystem instead of artifact API (avoids network upload latency) / Общая файловая система runner'а вместо artifact API
- Remove Astra Linux build temporarily / Сборка для Astra Linux временно удалена
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.20] - 2026-02-23
### Added / Добавлено
@@ -463,19 +395,6 @@ All notable changes to NORA will be documented in this file.
### Fixed / Исправлено
- Auth: replace `starts_with` with explicit `matches!` for token path checks / Аутентификация: `starts_with` заменён явной проверкой `matches!` для путей с токенами
- Remove unnecessary QEMU step for amd64-only builds / Удалён лишний шаг QEMU для amd64-сборок
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.19] - 2026-01-31
### Added / Добавлено
@@ -487,126 +406,47 @@ All notable changes to NORA will be documented in this file.
### Fixed / Исправлено
- Use `div_ceil` instead of manual ceiling division / Использован `div_ceil` вместо ручной реализации деления с округлением вверх
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.18] - 2026-01-31
### Changed
- Logo styling refinements
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.17] - 2026-01-31
### Added
- Copyright headers to all source files (Volkov Pavel | DevITWay)
- SPDX-License-Identifier: MIT in all .rs files
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.16] - 2026-01-31
### Changed
- N○RA branding: stylized O logo across dashboard
- Fixed O letter alignment in logo
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.15] - 2026-01-31
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Code formatting (cargo fmt)
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.14] - 2026-01-31
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Docker dashboard now shows actual image size from manifest layers (config + layers sum)
- Previously showed only manifest file size (~500 B instead of actual image size)
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.13] - 2026-01-31
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- npm dashboard now shows correct version count and package sizes
- Parses metadata.json for versions, dist.unpackedSize, and time.modified
- Previously showed 0 versions / 0 B for all packages
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.12] - 2026-01-30
### Added
@@ -628,106 +468,28 @@ All notable changes to NORA will be documented in this file.
#### Documentation
- Bilingual onboarding guide (EN/RU)
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.11] - 2026-01-26
### Added
- Internationalization (i18n) support
- PyPI registry proxy
- UI improvements
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.10] - 2026-01-26
### Changed
- Dark theme applied to all UI pages
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.9] - 2026-01-26
### Changed
- Version bump release
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.8] - 2026-01-26
### Added
- Dashboard endpoint added to OpenAPI documentation
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.7] - 2026-01-26
### Added
- Dynamic version display in UI sidebar
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.6] - 2026-01-26
### Added
@@ -739,54 +501,23 @@ All notable changes to NORA will be documented in this file.
#### UI
- Dark theme (bg: #0f172a, cards: #1e293b)
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.5] - 2026-01-26
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Docker push/pull: added PATCH endpoint for chunked uploads
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.4] - 2026-01-26
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Rate limiting: health/metrics endpoints now exempt
- Increased upload rate limits for Docker parallel requests
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.0] - 2026-01-25
### Added
@@ -854,19 +585,6 @@ All notable changes to NORA will be documented in this file.
- `src/error.rs` - application error types
- `src/request_id.rs` - request ID middleware
- `src/rate_limit.rs` - rate limiting configuration
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.1.0] - 2026-01-24
### Added
@@ -883,308 +601,3 @@ All notable changes to NORA will be documented in this file.
- Environment variable configuration
- Graceful shutdown (SIGTERM/SIGINT)
- Backup/restore commands
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
# Журнал изменений (RU)
Все значимые изменения NORA документируются в этом файле.
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.12] - 2026-01-30
### Добавлено
#### Настраиваемый Rate Limiting
- Rate limits настраиваются через `config.toml` и переменные окружения
- Новая секция `[rate_limit]` с параметрами: `auth_rps`, `auth_burst`, `upload_rps`, `upload_burst`, `general_rps`, `general_burst`
- Переменные окружения: `NORA_RATE_LIMIT_{AUTH|UPLOAD|GENERAL}_{RPS|BURST}`
#### Архитектура Secrets Provider
- Trait-based управление секретами (`SecretsProvider` trait)
- ENV provider по умолчанию (12-Factor App паттерн)
- Защищённые секреты с `zeroize` (память обнуляется при drop)
- Redacted Debug impl предотвращает утечку секретов в логи
- Новая секция `[secrets]` с опциями `provider` и `clear_env`
#### Docker Image Metadata
- Поддержка получения метаданных образов
#### Документация
- Двуязычный onboarding guide (EN/RU)
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.11] - 2026-01-26
### Добавлено
- Поддержка интернационализации (i18n)
- PyPI registry proxy
- Улучшения UI
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.10] - 2026-01-26
### Изменено
- Тёмная тема применена ко всем страницам UI
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.9] - 2026-01-26
### Изменено
- Релиз с обновлением версии
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.8] - 2026-01-26
### Добавлено
- Dashboard endpoint добавлен в OpenAPI документацию
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.7] - 2026-01-26
### Добавлено
- Динамическое отображение версии в сайдбаре UI
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.6] - 2026-01-26
### Добавлено
#### Dashboard Metrics
- Глобальная панель статистики: downloads, uploads, artifacts, cache hit rate, storage
- Расширенные карточки реестров с количеством артефактов, размером, счётчиками
- Лог активности (последние 20 событий)
#### UI
- Тёмная тема (bg: #0f172a, cards: #1e293b)
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.5] - 2026-01-26
### Исправлено
- Docker push/pull: добавлен PATCH endpoint для chunked uploads
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.4] - 2026-01-26
### Исправлено
- Rate limiting: health/metrics endpoints теперь исключены
- Увеличены лимиты upload для параллельных Docker запросов
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.2.0] - 2026-01-25
### Добавлено
#### UI: SVG иконки брендов
- Эмоджи заменены на SVG иконки брендов (стиль Simple Icons)
- Docker, Maven, npm, Cargo, PyPI теперь отображаются как векторная графика
- Единый стиль иконок на дашборде, сайдбаре и страницах деталей
#### Тестовая инфраструктура
- Unit-тесты для LocalStorage (8 тестов): put/get, list, stat, health_check
- Unit-тесты для S3Storage с HTTP-мокированием wiremock (11 тестов)
- Интеграционные тесты auth/htpasswd (7 тестов)
- Тесты жизненного цикла токенов (11 тестов)
- Тесты валидации (21 тест)
- **Всего: 75 тестов проходят**
#### Безопасность: Валидация ввода (`validation.rs`)
- Защита от path traversal: отклоняет `../`, `..\\`, null-байты, абсолютные пути
- Валидация имён Docker-образов по спецификации OCI distribution
- Валидация дайджестов (`sha256:[64 hex]`, `sha512:[128 hex]`)
- Валидация тегов и ссылок Docker
- Ограничение длины ключей хранилища (макс. 1024 символа)
#### Безопасность: Rate Limiting (`rate_limit.rs`)
- Auth endpoints: 1 req/sec, burst 5 (защита от брутфорса)
- Upload endpoints: 10 req/sec, burst 20
- Общие endpoints: 100 req/sec, burst 200
- Использует `tower_governor` 0.8 с `PeerIpKeyExtractor`
#### Наблюдаемость: Отслеживание Request ID (`request_id.rs`)
- Заголовок `X-Request-ID` добавляется ко всем ответам
- Принимает upstream request ID или генерирует UUID v4
- Tracing spans включают request_id для корреляции логов
#### CLI: Команда миграции (`migrate.rs`)
- `nora migrate --from local --to s3` - миграция между storage backends
- Флаг `--dry-run` для предпросмотра без копирования
- Прогресс-бар с indicatif
- Пропуск существующих файлов в destination
- Итоговая статистика (migrated, skipped, failed, bytes)
#### Обработка ошибок (`error.rs`)
- Enum `AppError` с `IntoResponse` для Axum
- Автоматическая конверсия из `StorageError` и `ValidationError`
- JSON-ответы об ошибках с поддержкой request_id
### Изменено
- `StorageError` теперь использует макрос `thiserror`
- `TokenError` теперь использует макрос `thiserror`
- Storage wrapper валидирует ключи перед делегированием backend
- Docker registry handlers валидируют name, digest, reference
- Лимит размера body установлен в 100MB через `DefaultBodyLimit`
### Добавлены зависимости
- `thiserror = "2"` - типизированная обработка ошибок
- `tower_governor = "0.8"` - rate limiting
- `governor = "0.10"` - backend для rate limiting
- `tempfile = "3"` (dev) - временные директории для тестов
- `wiremock = "0.6"` (dev) - HTTP-мокирование для S3 тестов
### Добавлены файлы
- `src/validation.rs` - модуль валидации ввода
- `src/migrate.rs` - модуль миграции хранилища
- `src/error.rs` - типы ошибок приложения
- `src/request_id.rs` - middleware для request ID
- `src/rate_limit.rs` - конфигурация rate limiting
---
## [0.2.30] - 2026-03-16
### Fixed / Исправлено
- **Dashboard**: Docker upstream now shown in mount points table (was null)
- **Dashboard**: Docker namespaced repositories (library/alpine, grafana/grafana) now visible in UI
- **Dashboard**: npm proxy-cached packages now appear in package list
- **Dashboard**: Отображение Docker upstream в таблице точек монтирования (было null)
- **Dashboard**: Namespaced Docker-репозитории (library/alpine, grafana/grafana) теперь видны в UI
- **Dashboard**: npm-пакеты из прокси-кеша теперь отображаются в списке пакетов
## [0.1.0] - 2026-01-24
### Добавлено
- Мульти-протокольная поддержка: Docker Registry v2, Maven, npm, Cargo, PyPI
- Web UI дашборд
- Swagger UI (`/api-docs`)
- Storage backends: локальная файловая система, S3-совместимое хранилище
- Умный прокси/кэш для Maven и npm
- Health checks (`/health`, `/ready`)
- Базовая аутентификация (htpasswd с bcrypt)
- API токены (отзываемые, per-user)
- Prometheus метрики (`/metrics`)
- JSON структурированное логирование
- Конфигурация через переменные окружения
- Graceful shutdown (SIGTERM/SIGINT)
- Команды backup/restore

149
COMPAT.md Normal file
View File

@@ -0,0 +1,149 @@
# NORA Registry Protocol Compatibility
This document describes which parts of each registry protocol are implemented in NORA.
**Legend:** Full = complete implementation, Partial = basic support with limitations, Stub = placeholder, — = not implemented
## Docker (OCI Distribution Spec 1.1)
| Endpoint | Method | Status | Notes |
|----------|--------|--------|-------|
| `/v2/` | GET | Full | API version check |
| `/v2/_catalog` | GET | Full | List all repositories |
| `/v2/{name}/tags/list` | GET | Full | List image tags |
| `/v2/{name}/manifests/{ref}` | GET | Full | By tag or digest |
| `/v2/{name}/manifests/{ref}` | HEAD | Full | Check manifest exists |
| `/v2/{name}/manifests/{ref}` | PUT | Full | Push manifest |
| `/v2/{name}/manifests/{ref}` | DELETE | Full | Delete manifest |
| `/v2/{name}/blobs/{digest}` | GET | Full | Download layer/config |
| `/v2/{name}/blobs/{digest}` | HEAD | Full | Check blob exists |
| `/v2/{name}/blobs/{digest}` | DELETE | Full | Delete blob |
| `/v2/{name}/blobs/uploads/` | POST | Full | Start chunked upload |
| `/v2/{name}/blobs/uploads/{uuid}` | PATCH | Full | Upload chunk |
| `/v2/{name}/blobs/uploads/{uuid}` | PUT | Full | Complete upload |
| Namespaced `{ns}/{name}` | * | Full | Two-level paths |
| Deep paths `a/b/c/name` | * | — | Max 2-level (`org/image`) |
| Token auth (Bearer) | — | Full | WWW-Authenticate challenge |
| Cross-repo blob mount | POST | — | Not implemented |
| Referrers API | GET | — | OCI 1.1 referrers |
### Known Limitations
- Max 2-level image path: `org/image:tag` works, `org/sub/path/image:tag` returns 404
- Large monolithic blob PUT (>~500MB) may fail even with high body limit
- No cross-repository blob mounting
## npm
| Feature | Status | Notes |
|---------|--------|-------|
| Package metadata (GET) | Full | JSON with all versions |
| Scoped packages `@scope/name` | Full | URL-encoded path |
| Tarball download | Full | SHA256 verified |
| Tarball URL rewriting | Full | Points to NORA, not upstream |
| Publish (`npm publish`) | Full | Immutable versions |
| Unpublish | — | Not implemented |
| Dist-tags (`latest`, `next`) | Partial | Read from metadata, no explicit management |
| Search (`/-/v1/search`) | — | Not implemented |
| Audit (`/-/npm/v1/security/advisories`) | — | Not implemented |
| Upstream proxy | Full | Configurable TTL |
## Maven
| Feature | Status | Notes |
|---------|--------|-------|
| Artifact download (GET) | Full | JAR, POM, checksums |
| Artifact upload (PUT) | Full | Any file type |
| GroupId path layout | Full | Dots → slashes |
| SHA1/MD5 checksums | Full | Stored alongside artifacts |
| `maven-metadata.xml` | Partial | Stored as-is, no auto-generation |
| SNAPSHOT versions | — | No SNAPSHOT resolution |
| Multi-proxy fallback | Full | Tries proxies in order |
| Content-Type by extension | Full | .jar, .pom, .xml, .sha1, .md5 |
### Known Limitations
- `maven-metadata.xml` not auto-generated on publish (must be uploaded explicitly)
- No SNAPSHOT version management (`-SNAPSHOT` → latest timestamp)
## Cargo (Sparse Index, RFC 2789)
| Feature | Status | Notes |
|---------|--------|-------|
| `config.json` | Full | `dl` and `api` fields |
| Sparse index lookup | Full | Prefix rules (1/2/3/ab/cd) |
| Crate download | Full | `.crate` files by version |
| `cargo publish` | Full | Length-prefixed JSON + .crate |
| Dependency metadata | Full | `req`, `package` transforms |
| SHA256 verification | Full | On publish |
| Cache-Control headers | Full | `immutable` for downloads, `max-age=300` for index |
| Yank/unyank | — | Not implemented |
| Owner management | — | Not implemented |
| Categories/keywords | Partial | Stored but not searchable |
## PyPI (PEP 503/691)
| Feature | Status | Notes |
|---------|--------|-------|
| Simple index (HTML) | Full | PEP 503 |
| Simple index (JSON) | Full | PEP 691, via Accept header |
| Package versions page | Full | HTML + JSON |
| File download | Full | Wheel, sdist, egg |
| `twine upload` | Full | Multipart form-data |
| SHA256 hashes | Full | In metadata links |
| Case normalization | Full | `My-Package``my-package` |
| Upstream proxy | Full | Configurable TTL |
| JSON API metadata | Full | `application/vnd.pypi.simple.v1+json` |
| Yanking | — | Not implemented |
| Upload signatures (PGP) | — | Not implemented |
## Go Module Proxy (GOPROXY)
| Feature | Status | Notes |
|---------|--------|-------|
| `/@v/list` | Full | List known versions |
| `/@v/{version}.info` | Full | Version metadata JSON |
| `/@v/{version}.mod` | Full | go.mod file |
| `/@v/{version}.zip` | Full | Module zip archive |
| `/@latest` | Full | Latest version info |
| Module path escaping | Full | `!x``X` per spec |
| Immutability | Full | .info, .mod, .zip immutable after first write |
| Size limit for .zip | Full | Configurable |
| `$GONOSUMDB` / `$GONOSUMCHECK` | — | Not relevant (client-side) |
| Upstream proxy | — | Direct storage only |
## Raw File Storage
| Feature | Status | Notes |
|---------|--------|-------|
| Upload (PUT) | Full | Any file type |
| Download (GET) | Full | Content-Type by extension |
| Delete (DELETE) | Full | |
| Exists check (HEAD) | Full | Returns size + Content-Type |
| Max file size | Full | Configurable (default 1MB) |
| Directory listing | — | Not implemented |
| Versioning | — | Overwrite-only |
## Helm OCI
Helm charts are stored as OCI artifacts via the Docker registry endpoints. `helm push` and `helm pull` work through the standard `/v2/` API.
| Feature | Status | Notes |
|---------|--------|-------|
| `helm push` (OCI) | Full | Via Docker PUT manifest/blob |
| `helm pull` (OCI) | Full | Via Docker GET manifest/blob |
| Helm repo index (`index.yaml`) | — | Not implemented (OCI only) |
## Cross-Cutting Features
| Feature | Status | Notes |
|---------|--------|-------|
| Authentication (Bearer/Basic) | Full | Per-request token validation |
| Anonymous read | Full | `NORA_AUTH_ANONYMOUS_READ=true` |
| Rate limiting | Full | `tower_governor`, per-IP |
| Prometheus metrics | Full | `/metrics` endpoint |
| Health check | Full | `/health` |
| Swagger/OpenAPI | Full | `/swagger-ui/` |
| S3 backend | Full | AWS, MinIO, any S3-compatible |
| Local filesystem backend | Full | Default, content-addressable |
| Activity log | Full | Recent push/pull in dashboard |
| Backup/restore | Full | CLI commands |
| Mirror CLI | Full | `nora mirror` for npm/pip/cargo/maven/docker |

View File

@@ -107,16 +107,19 @@ All three must pass. CI will enforce this.
- Run `cargo fmt` before committing
- Fix all `cargo clippy` warnings
- No `unwrap()` in production code (use proper error handling)
- Follow Rust naming conventions
- Keep functions short and focused
- Add tests for new functionality
## Pull Request Process
1. Update CHANGELOG.md if the change is user-facing
2. Add tests for new features or bug fixes
3. Ensure CI passes (fmt, clippy, test, security checks)
4. Keep PRs focused — one feature or fix per PR
1. Branch from `main`, use descriptive branch names (`feat/`, `fix/`, `chore/`)
2. Update CHANGELOG.md if the change is user-facing
3. Add tests for new features or bug fixes
4. Ensure CI passes (fmt, clippy, test, security checks)
5. Keep PRs focused — one feature or fix per PR
6. PRs are squash-merged to keep a clean history
## Commit Messages
@@ -131,6 +134,20 @@ Use conventional commits:
Example: `feat: add npm scoped package support`
## New Registry Checklist
When adding a new registry type (Docker, npm, Maven, etc.), ensure all of the following:
- [ ] Handler in `nora-registry/src/registry/`
- [ ] Health check endpoint
- [ ] Metrics (Prometheus)
- [ ] OpenAPI spec update
- [ ] Startup log line
- [ ] Dashboard UI tile
- [ ] Playwright e2e test
- [ ] CHANGELOG entry
- [ ] COMPAT.md update
## Reporting Issues
- Use GitHub Issues with the provided templates

106
Cargo.lock generated
View File

@@ -17,6 +17,15 @@ dependencies = [
"memchr",
]
[[package]]
name = "alloca"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5a7d05ea6aea7e9e64d25b9156ba2fee3fdd659e34e41063cd2fc7cd020d7f4"
dependencies = [
"cc",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
@@ -167,6 +176,7 @@ dependencies = [
"matchit",
"memchr",
"mime",
"multer",
"percent-encoding",
"pin-project-lite",
"serde_core",
@@ -488,25 +498,24 @@ dependencies = [
[[package]]
name = "criterion"
version = "0.5.1"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2b12d017a929603d80db1831cd3a24082f8137ce19c69e6447f54f5fc8d692f"
checksum = "950046b2aa2492f9a536f5f4f9a3de7b9e2476e575e05bd6c333371add4d98f3"
dependencies = [
"alloca",
"anes",
"cast",
"ciborium",
"clap",
"criterion-plot",
"is-terminal",
"itertools",
"num-traits",
"once_cell",
"oorandom",
"page_size",
"plotters",
"rayon",
"regex",
"serde",
"serde_derive",
"serde_json",
"tinytemplate",
"walkdir",
@@ -514,9 +523,9 @@ dependencies = [
[[package]]
name = "criterion-plot"
version = "0.5.0"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b50826342786a51a89e2da3a28f1c32b06e387201bc2d19791f622c673706b1"
checksum = "d8d80a2f4f5b554395e47b5d8305bc3d27813bacb73493eb1001e8f76dae29ea"
dependencies = [
"cast",
"itertools",
@@ -670,6 +679,15 @@ version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34aa73646ffb006b8f5147f3dc182bd4bcb190227ce861fc4a4844bf8e3cb2c0"
[[package]]
name = "encoding_rs"
version = "0.8.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75030f3c4f45dafd7586dd6780965a8c7e8e285a5ecb86713e63a79c5b2766f3"
dependencies = [
"cfg-if",
]
[[package]]
name = "equivalent"
version = "1.0.2"
@@ -1321,17 +1339,6 @@ dependencies = [
"serde",
]
[[package]]
name = "is-terminal"
version = "0.4.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3640c1c38b8e4e43584d8df18be5fc6b0aa314ce6ebf51b53313d4306cca8e46"
dependencies = [
"hermit-abi",
"libc",
"windows-sys 0.61.2",
]
[[package]]
name = "is_terminal_polyfill"
version = "1.70.2"
@@ -1340,9 +1347,9 @@ checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695"
[[package]]
name = "itertools"
version = "0.10.5"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b0fd2260e829bddf4cb6ea802289de2f86d6a7a690192fbe91b3f46e0f2c8473"
checksum = "413ee7dfc52ee1a4949ceeb7dbc8a33f2d6c088194d9f922fb8318faf1f01186"
dependencies = [
"either",
]
@@ -1503,6 +1510,23 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "multer"
version = "3.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83e87776546dc87511aa5ee218730c92b666d7264ab6ed41f9d215af9cd5224b"
dependencies = [
"bytes",
"encoding_rs",
"futures-util",
"http",
"httparse",
"memchr",
"mime",
"spin",
"version_check",
]
[[package]]
name = "nonempty"
version = "0.7.0"
@@ -1525,7 +1549,7 @@ dependencies = [
[[package]]
name = "nora-registry"
version = "0.4.0"
version = "0.5.0"
dependencies = [
"argon2",
"async-trait",
@@ -1614,6 +1638,16 @@ version = "11.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6790f58c7ff633d8771f42965289203411a5e5c68388703c06e14f24770b41e"
[[package]]
name = "page_size"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "30d5b2194ed13191c1999ae0704b7839fb18384fa22e49b57eeaa97d79ce40da"
dependencies = [
"libc",
"winapi",
]
[[package]]
name = "parking_lot"
version = "0.12.5"
@@ -2268,9 +2302,9 @@ dependencies = [
[[package]]
name = "serde_spanned"
version = "1.0.4"
version = "1.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8bbf91e5a4d6315eee45e704372590b30e260ee83af6639d64557f51b067776"
checksum = "6662b5879511e06e8999a8a235d848113e942c9124f211511b16466ee2995f26"
dependencies = [
"serde_core",
]
@@ -2362,6 +2396,12 @@ dependencies = [
"windows-sys 0.60.2",
]
[[package]]
name = "spin"
version = "0.9.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6980e8d7511241f8acf4aebddbb1ff938df5eebe98691418c4468d0b72a96a67"
[[package]]
name = "spinning_top"
version = "0.3.0"
@@ -2592,9 +2632,9 @@ dependencies = [
[[package]]
name = "toml"
version = "1.0.6+spec-1.1.0"
version = "1.1.2+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "399b1124a3c9e16766831c6bba21e50192572cdd98706ea114f9502509686ffc"
checksum = "81f3d15e84cbcd896376e6730314d59fb5a87f31e4b038454184435cd57defee"
dependencies = [
"indexmap",
"serde_core",
@@ -2607,27 +2647,27 @@ dependencies = [
[[package]]
name = "toml_datetime"
version = "1.0.0+spec-1.1.0"
version = "1.1.1+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32c2555c699578a4f59f0cc68e5116c8d7cabbd45e1409b989d4be085b53f13e"
checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7"
dependencies = [
"serde_core",
]
[[package]]
name = "toml_parser"
version = "1.0.9+spec-1.1.0"
version = "1.1.2+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "702d4415e08923e7e1ef96cd5727c0dfed80b4d2fa25db9647fe5eb6f7c5a4c4"
checksum = "a2abe9b86193656635d2411dc43050282ca48aa31c2451210f4202550afb7526"
dependencies = [
"winnow",
]
[[package]]
name = "toml_writer"
version = "1.0.6+spec-1.1.0"
version = "1.1.1+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab16f14aed21ee8bfd8ec22513f7287cd4a91aa92e44edfe2c17ddd004e92607"
checksum = "756daf9b1013ebe47a8776667b466417e2d4c5679d441c26230efd9ef78692db"
[[package]]
name = "tonic"
@@ -3365,9 +3405,9 @@ checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650"
[[package]]
name = "winnow"
version = "0.7.14"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a5364e9d77fcdeeaa6062ced926ee3381faa2ee02d3eb83a5c27a8825540829"
checksum = "09dac053f1cd375980747450bfc7250c264eaae0583872e845c0c7cd578872b5"
[[package]]
name = "wiremock"

View File

@@ -6,7 +6,7 @@ members = [
]
[workspace.package]
version = "0.4.0"
version = "0.5.0"
edition = "2021"
rust-version = "1.75"
license = "MIT"
@@ -16,7 +16,7 @@ homepage = "https://getnora.io"
[workspace.dependencies]
tokio = { version = "1", features = ["full"] }
axum = "0.8"
axum = { version = "0.8", features = ["multipart"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tracing = "0.1"
@@ -26,3 +26,27 @@ sha2 = "0.11"
async-trait = "0.1"
hmac = "0.13"
hex = "0.4"
[workspace.lints.clippy]
or_fun_call = "deny"
redundant_clone = "deny"
collection_is_never_read = "deny"
naive_bytecount = "deny"
stable_sort_primitive = "deny"
large_types_passed_by_value = "deny"
assigning_clones = "deny"
[workspace.lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = [] }
[profile.release]
debug = "line-tables-only"
codegen-units = 4
panic = "abort"
lto = "thin"
# Maximum optimization for GitHub Releases and published binaries
[profile.release-official]
inherits = "release"
codegen-units = 1
lto = true

View File

@@ -1,7 +1,7 @@
# syntax=docker/dockerfile:1.4
# NORA on Astra Linux SE base (Debian-based, FSTEC-certified)
# Binary is pre-built by CI and passed via context
FROM debian:bookworm-slim
FROM debian:bookworm-slim@sha256:f06537653ac770703bc45b4b113475bd402f451e85223f0f2837acbf89ab020a
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl \

View File

@@ -1,7 +1,7 @@
# syntax=docker/dockerfile:1.4
# NORA on RED OS base (RPM-based, FSTEC-certified)
# Binary is pre-built by CI and passed via context
FROM registry.access.redhat.com/ubi9/ubi-minimal:9.4
FROM registry.access.redhat.com/ubi9/ubi-minimal:9.4@sha256:c0e70387664f30cd9cf2795b547e4a9a51002c44a4a86aa9335ab030134bf392
RUN microdnf install -y ca-certificates shadow-utils \
&& microdnf clean all \

View File

@@ -9,7 +9,7 @@ docker run -d -p 4000:4000 -v nora-data:/data ghcr.io/getnora-io/nora:latest
Open [http://localhost:4000/ui/](http://localhost:4000/ui/) — your registry is ready.
<p align="center">
<img src=".github/assets/dashboard.gif" alt="NORA Dashboard" width="960" />
<img src=".github/assets/dashboard.png" alt="NORA Dashboard" width="960" />
</p>
## Why NORA

10
_typos.toml Normal file
View File

@@ -0,0 +1,10 @@
[default.extend-words]
# HashiCorp is not a typo
Hashi = "Hashi"
# flate2 is a Rust crate for compression
flate = "flate"
# grep pattern fragment in lint script
validat = "validat"
[files]
extend-exclude = ["vendor/", "*.lock", "target/", "fuzz/corpus/"]

6
artifacthub-repo.yml Normal file
View File

@@ -0,0 +1,6 @@
# Artifact Hub repository metadata
# https://artifacthub.io/docs/topics/repositories/
repositoryID: null # filled by Artifact Hub after registration
owners:
- name: DevITWay
email: devitway@gmail.com

View File

@@ -30,7 +30,7 @@ sha2.workspace = true
async-trait.workspace = true
hmac.workspace = true
hex.workspace = true
toml = "1.0"
toml = "1.1"
uuid = { version = "1", features = ["v4"] }
bcrypt = "0.19"
base64 = "0.22"
@@ -57,10 +57,13 @@ percent-encoding = "2"
proptest = "1"
tempfile = "3"
wiremock = "0.6"
criterion = { version = "0.5", features = ["html_reports"] }
criterion = { version = "0.8", features = ["html_reports"] }
tower = { version = "0.5", features = ["util"] }
http-body-util = "0.1"
[[bench]]
name = "parsing"
harness = false
[lints]
workspace = true

View File

@@ -28,6 +28,8 @@ pub struct Config {
#[serde(default)]
pub go: GoConfig,
#[serde(default)]
pub cargo: CargoConfig,
#[serde(default)]
pub raw: RawConfig,
#[serde(default)]
pub auth: AuthConfig,
@@ -129,6 +131,32 @@ pub struct PypiConfig {
pub proxy_timeout: u64,
}
/// Cargo registry configuration
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CargoConfig {
/// Upstream Cargo registry (crates.io API)
#[serde(default = "default_cargo_proxy")]
pub proxy: Option<String>,
#[serde(default)]
pub proxy_auth: Option<String>,
#[serde(default = "default_timeout")]
pub proxy_timeout: u64,
}
fn default_cargo_proxy() -> Option<String> {
Some("https://crates.io".to_string())
}
impl Default for CargoConfig {
fn default() -> Self {
Self {
proxy: default_cargo_proxy(),
proxy_auth: None,
proxy_timeout: 30,
}
}
}
/// Go module proxy configuration (GOPROXY protocol)
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GoConfig {
@@ -446,6 +474,10 @@ impl Config {
if self.pypi.proxy_auth.is_some() && std::env::var("NORA_PYPI_PROXY_AUTH").is_err() {
tracing::warn!("PyPI proxy credentials in config.toml are plaintext — consider NORA_PYPI_PROXY_AUTH env var");
}
// Cargo
if self.cargo.proxy_auth.is_some() && std::env::var("NORA_CARGO_PROXY_AUTH").is_err() {
tracing::warn!("Cargo proxy credentials in config.toml are plaintext — consider NORA_CARGO_PROXY_AUTH env var");
}
}
/// Validate configuration and return (warnings, errors).
@@ -703,6 +735,19 @@ impl Config {
}
}
// Cargo config
if let Ok(val) = env::var("NORA_CARGO_PROXY") {
self.cargo.proxy = if val.is_empty() { None } else { Some(val) };
}
if let Ok(val) = env::var("NORA_CARGO_PROXY_TIMEOUT") {
if let Ok(timeout) = val.parse() {
self.cargo.proxy_timeout = timeout;
}
}
if let Ok(val) = env::var("NORA_CARGO_PROXY_AUTH") {
self.cargo.proxy_auth = if val.is_empty() { None } else { Some(val) };
}
// Raw config
if let Ok(val) = env::var("NORA_RAW_ENABLED") {
self.raw.enabled = val.to_lowercase() == "true" || val == "1";
@@ -785,6 +830,7 @@ impl Default for Config {
npm: NpmConfig::default(),
pypi: PypiConfig::default(),
go: GoConfig::default(),
cargo: CargoConfig::default(),
docker: DockerConfig::default(),
raw: RawConfig::default(),
auth: AuthConfig::default(),
@@ -1371,4 +1417,11 @@ mod tests {
assert_eq!(config.go.proxy_auth, Some("user:pass".to_string()));
std::env::remove_var("NORA_GO_PROXY_AUTH");
}
#[test]
fn test_cargo_config_default() {
let c = CargoConfig::default();
assert_eq!(c.proxy, Some("https://crates.io".to_string()));
assert_eq!(c.proxy_timeout, 30);
}
}

View File

@@ -212,15 +212,16 @@ mod tests {
#[test]
fn test_record_download_all_registries() {
let m = DashboardMetrics::new();
for reg in &["docker", "npm", "maven", "cargo", "pypi", "raw"] {
for reg in &["docker", "npm", "maven", "cargo", "pypi", "go", "raw"] {
m.record_download(reg);
}
assert_eq!(m.downloads.load(Ordering::Relaxed), 6);
assert_eq!(m.downloads.load(Ordering::Relaxed), 7);
assert_eq!(m.get_registry_downloads("docker"), 1);
assert_eq!(m.get_registry_downloads("npm"), 1);
assert_eq!(m.get_registry_downloads("maven"), 1);
assert_eq!(m.get_registry_downloads("cargo"), 1);
assert_eq!(m.get_registry_downloads("pypi"), 1);
assert_eq!(m.get_registry_downloads("go"), 1);
assert_eq!(m.get_registry_downloads("raw"), 1);
}
@@ -233,14 +234,18 @@ mod tests {
}
#[test]
fn test_record_upload() {
fn test_record_upload_all_registries() {
let m = DashboardMetrics::new();
m.record_upload("docker");
m.record_upload("maven");
m.record_upload("raw");
assert_eq!(m.uploads.load(Ordering::Relaxed), 3);
for reg in &["docker", "npm", "maven", "cargo", "pypi", "go", "raw"] {
m.record_upload(reg);
}
assert_eq!(m.uploads.load(Ordering::Relaxed), 7);
assert_eq!(m.get_registry_uploads("docker"), 1);
assert_eq!(m.get_registry_uploads("npm"), 1);
assert_eq!(m.get_registry_uploads("maven"), 1);
assert_eq!(m.get_registry_uploads("cargo"), 1);
assert_eq!(m.get_registry_uploads("pypi"), 1);
assert_eq!(m.get_registry_uploads("go"), 1);
assert_eq!(m.get_registry_uploads("raw"), 1);
}

View File

@@ -1,10 +1,14 @@
//! Garbage Collection for orphaned blobs
//! Garbage Collection for orphaned Docker blobs
//!
//! Mark-and-sweep approach:
//! 1. List all blobs across registries
//! 2. Parse all manifests to find referenced blobs
//! 1. List all Docker blobs
//! 2. Parse Docker manifests to find referenced blobs
//! 3. Blobs not referenced by any manifest = orphans
//! 4. Delete orphans (with --dry-run support)
//!
//! Currently Docker-only. Other registries (npm, maven, cargo, pypi, go,
//! raw) are excluded because no reference resolver exists for their
//! metadata formats.
use std::collections::HashSet;
@@ -72,17 +76,17 @@ pub async fn run_gc(storage: &Storage, dry_run: bool) -> GcResult {
async fn collect_all_blobs(storage: &Storage) -> Vec<String> {
let mut blobs = Vec::new();
// Collect blobs from all registry types, not just Docker
for prefix in &[
"docker/", "maven/", "npm/", "cargo/", "pypi/", "raw/", "go/",
] {
let keys = storage.list(prefix).await;
// Only collect Docker blobs. Other registries (npm, maven, cargo, pypi,
// go, raw) use storage key schemes that collect_referenced_digests does
// not understand, so their artifacts would appear as orphans and be
// deleted. Extending GC to non-Docker registries requires per-registry
// reference resolution.
let keys = storage.list("docker/").await;
for key in keys {
if key.contains("/blobs/") || key.contains("/tarballs/") {
if key.contains("/blobs/") {
blobs.push(key);
}
}
}
blobs
}
@@ -304,18 +308,103 @@ mod tests {
}
#[tokio::test]
async fn test_gc_multi_registry_blobs() {
async fn test_gc_ignores_non_docker_registries() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// npm tarball (not referenced by Docker manifests => orphan candidate)
// Non-Docker artifacts must not be collected by GC, because
// collect_referenced_digests only understands Docker manifests.
// Without this guard, these would all appear as orphans and be deleted.
storage
.put("npm/lodash/tarballs/lodash-4.17.21.tgz", b"tarball-data")
.await
.unwrap();
storage
.put("maven/com/example/lib/1.0/lib-1.0.jar", b"jar-data")
.await
.unwrap();
storage
.put("cargo/serde/1.0.0/serde-1.0.0.crate", b"crate-data")
.await
.unwrap();
let result = run_gc(&storage, true).await;
assert_eq!(result.total_blobs, 0);
assert_eq!(result.orphaned_blobs, 0);
}
#[tokio::test]
async fn test_gc_does_not_delete_npm_tarballs() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// Regression test: npm tarballs were previously collected because
// their keys contain "/tarballs/", but no reference resolver existed
// for npm metadata, so they were all treated as orphans.
storage
.put("npm/lodash/tarballs/lodash-4.17.21.tgz", b"tarball-data")
.await
.unwrap();
let result = run_gc(&storage, true).await;
// npm tarballs contain "tarballs/" which matches the filter
assert_eq!(result.total_blobs, 1);
let result = run_gc(&storage, false).await;
assert_eq!(result.deleted_blobs, 0);
// Verify tarball still exists
assert!(storage
.get("npm/lodash/tarballs/lodash-4.17.21.tgz")
.await
.is_ok());
}
#[tokio::test]
async fn test_gc_deletes_docker_orphan_but_preserves_npm() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// Docker manifest referencing one blob
let manifest = serde_json::json!({
"config": {"digest": "sha256:configabc"},
"layers": []
});
storage
.put(
"docker/test/manifests/latest.json",
manifest.to_string().as_bytes(),
)
.await
.unwrap();
storage
.put("docker/test/blobs/sha256:configabc", b"config")
.await
.unwrap();
// Orphan Docker blob
storage
.put("docker/test/blobs/sha256:orphan1", b"orphan")
.await
.unwrap();
// npm tarball that must survive
storage
.put("npm/lodash/tarballs/lodash-4.17.21.tgz", b"tarball-data")
.await
.unwrap();
let result = run_gc(&storage, false).await;
assert_eq!(result.total_blobs, 2); // only Docker blobs counted
assert_eq!(result.orphaned_blobs, 1);
assert_eq!(result.deleted_blobs, 1);
// Docker orphan gone
assert!(storage
.get("docker/test/blobs/sha256:orphan1")
.await
.is_err());
// Docker referenced blob still exists
assert!(storage
.get("docker/test/blobs/sha256:configabc")
.await
.is_ok());
// npm tarball untouched
assert!(storage
.get("npm/lodash/tarballs/lodash-4.17.21.tgz")
.await
.is_ok());
}
}

View File

@@ -31,6 +31,8 @@ pub struct RegistriesHealth {
pub npm: String,
pub cargo: String,
pub pypi: String,
pub go: String,
pub raw: String,
}
pub fn routes() -> Router<Arc<AppState>> {
@@ -70,6 +72,8 @@ async fn health_check(State(state): State<Arc<AppState>>) -> (StatusCode, Json<H
npm: "ok".to_string(),
cargo: "ok".to_string(),
pypi: "ok".to_string(),
go: "ok".to_string(),
raw: "ok".to_string(),
},
};

View File

@@ -438,6 +438,7 @@ async fn run_server(config: Config, storage: Storage) {
npm = "/npm/",
cargo = "/cargo/",
pypi = "/simple/",
go = "/go/",
raw = "/raw/",
"Available endpoints"
);

View File

@@ -121,6 +121,10 @@ fn detect_registry(path: &str) -> String {
"cargo".to_string()
} else if path.starts_with("/simple") || path.starts_with("/packages") {
"pypi".to_string()
} else if path.starts_with("/go/") {
"go".to_string()
} else if path.starts_with("/raw/") {
"raw".to_string()
} else if path.starts_with("/ui") {
"ui".to_string()
} else {
@@ -205,8 +209,19 @@ mod tests {
fn test_detect_registry_go_path() {
assert_eq!(
detect_registry("/go/github.com/user/repo/@v/v1.0.0.info"),
"other"
"go"
);
assert_eq!(detect_registry("/go/github.com/user/repo/@latest"), "go");
// Bare prefix without trailing slash should not match
assert_eq!(detect_registry("/goblin/something"), "other");
}
#[test]
fn test_detect_registry_raw_path() {
assert_eq!(detect_registry("/raw/my-project/artifact.tar.gz"), "raw");
assert_eq!(detect_registry("/raw/data/file.bin"), "raw");
// Bare prefix without trailing slash should not match
assert_eq!(detect_registry("/rawdata/file"), "other");
}
#[test]

View File

@@ -118,6 +118,15 @@ pub async fn run_mirror(
packages,
all_versions,
} => {
if let Some(ref lf) = lockfile {
let content = std::fs::read_to_string(lf)
.map_err(|e| format!("Cannot read {}: {}", lf.display(), e))?;
if content.contains("# yarn lockfile v1")
|| content.starts_with("# THIS IS AN AUTOGENERATED FILE")
{
return Err("This looks like a yarn.lock file. Use `nora mirror yarn --lockfile` instead.".to_string());
}
}
npm::run_npm_mirror(
&client,
registry,
@@ -269,7 +278,19 @@ async fn mirror_lockfile(
}
fetched += 1;
}
_ => failed += 1,
Ok(r) => {
eprintln!(
" WARN: {} {} -> HTTP {}",
target.name,
target.version,
r.status()
);
failed += 1;
}
Err(e) => {
eprintln!(" WARN: {} {} -> {}", target.name, target.version, e);
failed += 1;
}
}
pb.set_message(format!("{}@{}", target.name, target.version));
@@ -277,6 +298,11 @@ async fn mirror_lockfile(
}
pb.finish_with_message("done");
if format == "pip" && fetched > 0 {
eprintln!(
" NOTE: Only top-level packages were mirrored. For air-gapped installs,\n use `pip freeze > requirements.txt` to include all transitive dependencies."
);
}
Ok(MirrorResult {
total: targets.len(),
fetched,

View File

@@ -18,8 +18,8 @@ use crate::AppState;
#[openapi(
info(
title = "Nora",
version = "0.2.12",
description = "Multi-protocol package registry supporting Docker, Maven, npm, Cargo, and PyPI",
version = "0.5.0",
description = "Multi-protocol package registry supporting Docker, Maven, npm, Cargo, PyPI, Go, and Raw",
license(name = "MIT"),
contact(name = "DevITWay", url = "https://github.com/getnora-io/nora")
),
@@ -35,6 +35,8 @@ use crate::AppState;
(name = "npm", description = "npm Registry API"),
(name = "cargo", description = "Cargo Registry API"),
(name = "pypi", description = "PyPI Simple API"),
(name = "go", description = "Go Module Proxy API"),
(name = "raw", description = "Raw File Storage API"),
(name = "auth", description = "Authentication & API Tokens")
),
paths(

File diff suppressed because it is too large Load Diff

View File

@@ -1,28 +1,47 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT
//! PyPI registry — PEP 503 (Simple HTML) + PEP 691 (JSON) + twine upload.
//!
//! Implements:
//! GET /simple/ — package index (HTML or JSON)
//! GET /simple/{name}/ — package versions (HTML or JSON)
//! GET /simple/{name}/{filename} — download file
//! POST /simple/ — twine upload (multipart/form-data)
use crate::activity_log::{ActionType, ActivityEntry};
use crate::audit::AuditEntry;
use crate::registry::{proxy_fetch, proxy_fetch_text};
use crate::AppState;
use axum::{
extract::{Path, State},
http::{header, StatusCode},
extract::{Multipart, Path, State},
http::{header, HeaderMap, StatusCode},
response::{Html, IntoResponse, Response},
routing::get,
Router,
};
use sha2::Digest;
use std::sync::Arc;
/// PEP 691 JSON content type
const PEP691_JSON: &str = "application/vnd.pypi.simple.v1+json";
pub fn routes() -> Router<Arc<AppState>> {
Router::new()
.route("/simple/", get(list_packages))
.route("/simple/", get(list_packages).post(upload))
.route("/simple/{name}/", get(package_versions))
.route("/simple/{name}/{filename}", get(download_file))
}
/// List all packages (Simple API index)
async fn list_packages(State(state): State<Arc<AppState>>) -> impl IntoResponse {
// ============================================================================
// Package index
// ============================================================================
/// GET /simple/ — list all packages (PEP 503 HTML or PEP 691 JSON).
async fn list_packages(
State(state): State<Arc<AppState>>,
headers: HeaderMap,
) -> impl IntoResponse {
let keys = state.storage.list("pypi/").await;
let mut packages = std::collections::HashSet::new();
@@ -34,52 +53,77 @@ async fn list_packages(State(state): State<Arc<AppState>>) -> impl IntoResponse
}
}
let mut html = String::from(
"<!DOCTYPE html>\n<html><head><title>Simple Index</title></head><body><h1>Simple Index</h1>\n",
);
let mut pkg_list: Vec<_> = packages.into_iter().collect();
pkg_list.sort();
if wants_json(&headers) {
// PEP 691 JSON response
let projects: Vec<serde_json::Value> = pkg_list
.iter()
.map(|name| serde_json::json!({"name": name}))
.collect();
let body = serde_json::json!({
"meta": {"api-version": "1.0"},
"projects": projects,
});
(
StatusCode::OK,
[(header::CONTENT_TYPE, PEP691_JSON)],
serde_json::to_string(&body).unwrap_or_default(),
)
.into_response()
} else {
// PEP 503 HTML
let mut html = String::from(
"<!DOCTYPE html>\n<html><head><title>Simple Index</title></head><body><h1>Simple Index</h1>\n",
);
for pkg in pkg_list {
html.push_str(&format!("<a href=\"/simple/{}/\">{}</a><br>\n", pkg, pkg));
}
html.push_str("</body></html>");
(StatusCode::OK, Html(html))
(StatusCode::OK, Html(html)).into_response()
}
}
/// List versions/files for a specific package
// ============================================================================
// Package versions
// ============================================================================
/// GET /simple/{name}/ — list files for a package (PEP 503 HTML or PEP 691 JSON).
async fn package_versions(
State(state): State<Arc<AppState>>,
Path(name): Path<String>,
headers: HeaderMap,
) -> Response {
// Normalize package name (PEP 503)
let normalized = normalize_name(&name);
// Try to get local files first
let prefix = format!("pypi/{}/", normalized);
let keys = state.storage.list(&prefix).await;
if !keys.is_empty() {
// We have local files
let mut html = format!(
"<!DOCTYPE html>\n<html><head><title>Links for {}</title></head><body><h1>Links for {}</h1>\n",
name, name
);
// Collect files with their hashes
let mut files: Vec<FileEntry> = Vec::new();
for key in &keys {
if let Some(filename) = key.strip_prefix(&prefix) {
if !filename.is_empty() {
html.push_str(&format!(
"<a href=\"/simple/{}/{}\">{}</a><br>\n",
normalized, filename, filename
));
if !filename.is_empty() && !filename.ends_with(".sha256") {
let sha256 = state
.storage
.get(&format!("{}.sha256", key))
.await
.ok()
.and_then(|d| String::from_utf8(d.to_vec()).ok());
files.push(FileEntry {
filename: filename.to_string(),
sha256,
});
}
}
}
html.push_str("</body></html>");
return (StatusCode::OK, Html(html)).into_response();
if !files.is_empty() {
return if wants_json(&headers) {
versions_json_response(&normalized, &files)
} else {
versions_html_response(&normalized, &files)
};
}
// Try proxy if configured
@@ -95,7 +139,6 @@ async fn package_versions(
)
.await
{
// Rewrite URLs in the HTML to point to our registry
let rewritten = rewrite_pypi_links(&html, &normalized);
return (StatusCode::OK, Html(rewritten)).into_response();
}
@@ -104,7 +147,11 @@ async fn package_versions(
StatusCode::NOT_FOUND.into_response()
}
/// Download a specific file
// ============================================================================
// Download
// ============================================================================
/// GET /simple/{name}/{filename} — download a specific file.
async fn download_file(
State(state): State<Arc<AppState>>,
Path((name, filename)): Path<(String, String)>,
@@ -126,20 +173,12 @@ async fn download_file(
.audit
.log(AuditEntry::new("cache_hit", "api", "", "pypi", ""));
let content_type = if filename.ends_with(".whl") {
"application/zip"
} else if filename.ends_with(".tar.gz") || filename.ends_with(".tgz") {
"application/gzip"
} else {
"application/octet-stream"
};
let content_type = pypi_content_type(&filename);
return (StatusCode::OK, [(header::CONTENT_TYPE, content_type)], data).into_response();
}
// Try proxy if configured
if let Some(proxy_url) = &state.config.pypi.proxy {
// First, fetch the package page to find the actual download URL
let page_url = format!("{}/{}/", proxy_url.trim_end_matches('/'), normalized);
if let Ok(html) = proxy_fetch_text(
@@ -151,7 +190,6 @@ async fn download_file(
)
.await
{
// Find the URL for this specific file
if let Some(file_url) = find_file_url(&html, &filename) {
if let Ok(data) = proxy_fetch(
&state.http_client,
@@ -173,24 +211,21 @@ async fn download_file(
.audit
.log(AuditEntry::new("proxy_fetch", "api", "", "pypi", ""));
// Cache in local storage
// Cache in background + compute hash
let storage = state.storage.clone();
let key_clone = key.clone();
let data_clone = data.clone();
tokio::spawn(async move {
let _ = storage.put(&key_clone, &data_clone).await;
let hash = hex::encode(sha2::Sha256::digest(&data_clone));
let _ = storage
.put(&format!("{}.sha256", key_clone), hash.as_bytes())
.await;
});
state.repo_index.invalidate("pypi");
let content_type = if filename.ends_with(".whl") {
"application/zip"
} else if filename.ends_with(".tar.gz") || filename.ends_with(".tgz") {
"application/gzip"
} else {
"application/octet-stream"
};
let content_type = pypi_content_type(&filename);
return (StatusCode::OK, [(header::CONTENT_TYPE, content_type)], data)
.into_response();
}
@@ -201,14 +236,238 @@ async fn download_file(
StatusCode::NOT_FOUND.into_response()
}
/// Normalize package name according to PEP 503
// ============================================================================
// Twine upload (PEP 503 — POST /simple/)
// ============================================================================
/// POST /simple/ — upload a package via twine.
///
/// twine sends multipart/form-data with fields:
/// :action = "file_upload"
/// name = package name
/// version = package version
/// filetype = "sdist" | "bdist_wheel"
/// content = the file bytes
/// sha256_digest = hex SHA-256 of file (optional)
/// metadata_version, summary, etc. (optional metadata)
async fn upload(State(state): State<Arc<AppState>>, mut multipart: Multipart) -> Response {
let mut action = String::new();
let mut name = String::new();
let mut version = String::new();
let mut filename = String::new();
let mut file_data: Option<Vec<u8>> = None;
let mut sha256_digest = String::new();
// Parse multipart fields
while let Ok(Some(field)) = multipart.next_field().await {
let field_name = field.name().unwrap_or("").to_string();
match field_name.as_str() {
":action" => {
action = field.text().await.ok().unwrap_or_default();
}
"name" => {
name = field.text().await.ok().unwrap_or_default();
}
"version" => {
version = field.text().await.ok().unwrap_or_default();
}
"sha256_digest" => {
sha256_digest = field.text().await.ok().unwrap_or_default();
}
"content" => {
filename = field.file_name().unwrap_or("unknown").to_string();
match field.bytes().await {
Ok(b) => file_data = Some(b.to_vec()),
Err(e) => {
return (
StatusCode::BAD_REQUEST,
format!("Failed to read file: {}", e),
)
.into_response()
}
}
}
_ => {
// Skip other metadata fields (summary, author, etc.)
let _ = field.bytes().await;
}
}
}
// Validate required fields
if action != "file_upload" {
return (StatusCode::BAD_REQUEST, "Unsupported action").into_response();
}
if name.is_empty() || version.is_empty() {
return (StatusCode::BAD_REQUEST, "Missing name or version").into_response();
}
let data = match file_data {
Some(d) if !d.is_empty() => d,
_ => return (StatusCode::BAD_REQUEST, "Missing file content").into_response(),
};
// Validate filename
if filename.is_empty() || !is_valid_pypi_filename(&filename) {
return (StatusCode::BAD_REQUEST, "Invalid filename").into_response();
}
// Verify SHA-256 if provided
let computed_hash = hex::encode(sha2::Sha256::digest(&data));
if !sha256_digest.is_empty() && sha256_digest != computed_hash {
tracing::warn!(
package = %name,
expected = %sha256_digest,
computed = %computed_hash,
"SECURITY: PyPI upload SHA-256 mismatch"
);
return (StatusCode::BAD_REQUEST, "SHA-256 digest mismatch").into_response();
}
// Normalize name and store
let normalized = normalize_name(&name);
// Check immutability (same filename = already exists)
let file_key = format!("pypi/{}/{}", normalized, filename);
if state.storage.stat(&file_key).await.is_some() {
return (
StatusCode::CONFLICT,
format!("File {} already exists", filename),
)
.into_response();
}
// Store file
if state.storage.put(&file_key, &data).await.is_err() {
return StatusCode::INTERNAL_SERVER_ERROR.into_response();
}
// Store SHA-256 hash
let hash_key = format!("{}.sha256", file_key);
let _ = state.storage.put(&hash_key, computed_hash.as_bytes()).await;
state.metrics.record_upload("pypi");
state.activity.push(ActivityEntry::new(
ActionType::Push,
format!("{}-{}", name, version),
"pypi",
"LOCAL",
));
state
.audit
.log(AuditEntry::new("push", "api", "", "pypi", ""));
state.repo_index.invalidate("pypi");
StatusCode::OK.into_response()
}
// ============================================================================
// PEP 691 JSON responses
// ============================================================================
struct FileEntry {
filename: String,
sha256: Option<String>,
}
fn versions_json_response(normalized: &str, files: &[FileEntry]) -> Response {
let file_entries: Vec<serde_json::Value> = files
.iter()
.map(|f| {
let mut entry = serde_json::json!({
"filename": f.filename,
"url": format!("/simple/{}/{}", normalized, f.filename),
});
if let Some(hash) = &f.sha256 {
entry["digests"] = serde_json::json!({"sha256": hash});
}
entry
})
.collect();
let body = serde_json::json!({
"meta": {"api-version": "1.0"},
"name": normalized,
"files": file_entries,
});
(
StatusCode::OK,
[(header::CONTENT_TYPE, PEP691_JSON)],
serde_json::to_string(&body).unwrap_or_default(),
)
.into_response()
}
fn versions_html_response(normalized: &str, files: &[FileEntry]) -> Response {
let mut html = format!(
"<!DOCTYPE html>\n<html><head><title>Links for {}</title></head><body><h1>Links for {}</h1>\n",
normalized, normalized
);
for f in files {
let hash_fragment = f
.sha256
.as_ref()
.map(|h| format!("#sha256={}", h))
.unwrap_or_default();
html.push_str(&format!(
"<a href=\"/simple/{}/{}{}\">{}</a><br>\n",
normalized, f.filename, hash_fragment, f.filename
));
}
html.push_str("</body></html>");
(StatusCode::OK, Html(html)).into_response()
}
// ============================================================================
// Helpers
// ============================================================================
/// Normalize package name according to PEP 503.
fn normalize_name(name: &str) -> String {
name.to_lowercase().replace(['-', '_', '.'], "-")
}
/// Rewrite PyPI links to point to our registry
/// Check Accept header for PEP 691 JSON.
fn wants_json(headers: &HeaderMap) -> bool {
headers
.get(header::ACCEPT)
.and_then(|v| v.to_str().ok())
.map(|v| v.contains(PEP691_JSON))
.unwrap_or(false)
}
/// Content-type for PyPI files.
fn pypi_content_type(filename: &str) -> &'static str {
if filename.ends_with(".whl") {
"application/zip"
} else if filename.ends_with(".tar.gz") || filename.ends_with(".tgz") {
"application/gzip"
} else {
"application/octet-stream"
}
}
/// Validate PyPI filename.
fn is_valid_pypi_filename(name: &str) -> bool {
!name.is_empty()
&& !name.contains("..")
&& !name.contains('/')
&& !name.contains('\\')
&& !name.contains('\0')
&& (name.ends_with(".tar.gz")
|| name.ends_with(".tgz")
|| name.ends_with(".whl")
|| name.ends_with(".zip")
|| name.ends_with(".egg"))
}
/// Rewrite PyPI links to point to our registry.
fn rewrite_pypi_links(html: &str, package_name: &str) -> String {
// Simple regex-free approach: find href="..." and rewrite
let mut result = String::with_capacity(html.len());
let mut remaining = html;
@@ -219,10 +478,13 @@ fn rewrite_pypi_links(html: &str, package_name: &str) -> String {
if let Some(href_end) = remaining.find('"') {
let url = &remaining[..href_end];
// Extract filename from URL
if let Some(filename) = extract_filename(url) {
// Rewrite to our local URL
result.push_str(&format!("/simple/{}/{}", package_name, filename));
// Extract hash fragment from original URL
let hash_fragment = url.find('#').map(|pos| &url[pos..]).unwrap_or("");
result.push_str(&format!(
"/simple/{}/{}{}",
package_name, filename, hash_fragment
));
} else {
result.push_str(url);
}
@@ -233,12 +495,11 @@ fn rewrite_pypi_links(html: &str, package_name: &str) -> String {
result.push_str(remaining);
// Remove data-core-metadata and data-dist-info-metadata attributes
// as we don't serve .metadata files (PEP 658)
let result = remove_attribute(&result, "data-core-metadata");
remove_attribute(&result, "data-dist-info-metadata")
}
/// Remove an HTML attribute from all tags
/// Remove an HTML attribute from all tags.
fn remove_attribute(html: &str, attr_name: &str) -> String {
let mut result = String::with_capacity(html.len());
let mut remaining = html;
@@ -248,7 +509,6 @@ fn remove_attribute(html: &str, attr_name: &str) -> String {
result.push_str(&remaining[..attr_start]);
remaining = &remaining[attr_start + pattern.len()..];
// Skip the attribute value
if let Some(attr_end) = remaining.find('"') {
remaining = &remaining[attr_end + 1..];
}
@@ -257,19 +517,11 @@ fn remove_attribute(html: &str, attr_name: &str) -> String {
result
}
/// Extract filename from PyPI download URL
/// Extract filename from PyPI download URL.
fn extract_filename(url: &str) -> Option<&str> {
// PyPI URLs look like:
// https://files.pythonhosted.org/packages/.../package-1.0.0.tar.gz#sha256=...
// or just the filename directly
// Remove hash fragment
let url = url.split('#').next()?;
// Get the last path component
let filename = url.rsplit('/').next()?;
// Must be a valid package file
if filename.ends_with(".tar.gz")
|| filename.ends_with(".tgz")
|| filename.ends_with(".whl")
@@ -282,7 +534,7 @@ fn extract_filename(url: &str) -> Option<&str> {
}
}
/// Find the download URL for a specific file in the HTML
/// Find the download URL for a specific file in the HTML.
fn find_file_url(html: &str, target_filename: &str) -> Option<String> {
let mut remaining = html;
@@ -294,7 +546,6 @@ fn find_file_url(html: &str, target_filename: &str) -> Option<String> {
if let Some(filename) = extract_filename(url) {
if filename == target_filename {
// Remove hash fragment for actual download
return Some(url.split('#').next().unwrap_or(url).to_string());
}
}
@@ -306,6 +557,10 @@ fn find_file_url(html: &str, target_filename: &str) -> Option<String> {
None
}
// ============================================================================
// Unit Tests
// ============================================================================
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
@@ -481,7 +736,14 @@ mod tests {
fn test_rewrite_pypi_links_basic() {
let html = r#"<a href="https://files.pythonhosted.org/packages/aa/bb/flask-2.0.tar.gz#sha256=abc">flask-2.0.tar.gz</a>"#;
let result = rewrite_pypi_links(html, "flask");
assert!(result.contains("/simple/flask/flask-2.0.tar.gz"));
assert!(result.contains("/simple/flask/flask-2.0.tar.gz#sha256=abc"));
}
#[test]
fn test_rewrite_pypi_links_preserves_hash() {
let html = r#"<a href="https://example.com/pkg-1.0.whl#sha256=deadbeef">pkg</a>"#;
let result = rewrite_pypi_links(html, "pkg");
assert!(result.contains("#sha256=deadbeef"));
}
#[test]
@@ -527,12 +789,50 @@ mod tests {
let result = find_file_url(html, "pkg-1.0.whl");
assert_eq!(result, Some("https://example.com/pkg-1.0.whl".to_string()));
}
#[test]
fn test_is_valid_pypi_filename() {
assert!(is_valid_pypi_filename("flask-2.0.tar.gz"));
assert!(is_valid_pypi_filename("flask-2.0-py3-none-any.whl"));
assert!(is_valid_pypi_filename("flask-2.0.tgz"));
assert!(is_valid_pypi_filename("flask-2.0.zip"));
assert!(is_valid_pypi_filename("flask-2.0.egg"));
assert!(!is_valid_pypi_filename(""));
assert!(!is_valid_pypi_filename("../evil.tar.gz"));
assert!(!is_valid_pypi_filename("evil/path.tar.gz"));
assert!(!is_valid_pypi_filename("noext"));
assert!(!is_valid_pypi_filename("bad.exe"));
}
#[test]
fn test_wants_json_pep691() {
let mut headers = HeaderMap::new();
headers.insert(header::ACCEPT, PEP691_JSON.parse().unwrap());
assert!(wants_json(&headers));
}
#[test]
fn test_wants_json_html() {
let mut headers = HeaderMap::new();
headers.insert(header::ACCEPT, "text/html".parse().unwrap());
assert!(!wants_json(&headers));
}
#[test]
fn test_wants_json_no_header() {
let headers = HeaderMap::new();
assert!(!wants_json(&headers));
}
}
// ============================================================================
// Integration Tests
// ============================================================================
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod integration_tests {
use crate::test_helpers::{body_bytes, create_test_context, send};
use crate::test_helpers::{body_bytes, create_test_context, send, send_with_headers};
use axum::http::{Method, StatusCode};
#[tokio::test]
@@ -550,7 +850,6 @@ mod integration_tests {
async fn test_pypi_list_with_packages() {
let ctx = create_test_context();
// Pre-populate storage with a package
ctx.state
.storage
.put("pypi/flask/flask-2.0.tar.gz", b"fake-tarball-data")
@@ -565,11 +864,36 @@ mod integration_tests {
assert!(html.contains("flask"));
}
#[tokio::test]
async fn test_pypi_list_json_pep691() {
let ctx = create_test_context();
ctx.state
.storage
.put("pypi/flask/flask-2.0.tar.gz", b"data")
.await
.unwrap();
let response = send_with_headers(
&ctx.app,
Method::GET,
"/simple/",
vec![("Accept", "application/vnd.pypi.simple.v1+json")],
"",
)
.await;
assert_eq!(response.status(), StatusCode::OK);
let body = body_bytes(response).await;
let json: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert!(json["meta"]["api-version"].as_str() == Some("1.0"));
assert!(json["projects"].as_array().unwrap().len() == 1);
}
#[tokio::test]
async fn test_pypi_versions_local() {
let ctx = create_test_context();
// Pre-populate storage
ctx.state
.storage
.put("pypi/flask/flask-2.0.tar.gz", b"fake-data")
@@ -585,6 +909,65 @@ mod integration_tests {
assert!(html.contains("/simple/flask/flask-2.0.tar.gz"));
}
#[tokio::test]
async fn test_pypi_versions_with_hash() {
let ctx = create_test_context();
ctx.state
.storage
.put("pypi/flask/flask-2.0.tar.gz", b"fake-data")
.await
.unwrap();
ctx.state
.storage
.put(
"pypi/flask/flask-2.0.tar.gz.sha256",
b"abc123def456abc123def456abc123def456abc123def456abc123def456abcd",
)
.await
.unwrap();
let response = send(&ctx.app, Method::GET, "/simple/flask/", "").await;
assert_eq!(response.status(), StatusCode::OK);
let body = body_bytes(response).await;
let html = String::from_utf8_lossy(&body);
assert!(html.contains("#sha256=abc123"));
}
#[tokio::test]
async fn test_pypi_versions_json_pep691() {
let ctx = create_test_context();
ctx.state
.storage
.put("pypi/flask/flask-2.0.tar.gz", b"data")
.await
.unwrap();
ctx.state
.storage
.put("pypi/flask/flask-2.0.tar.gz.sha256", b"deadbeef")
.await
.unwrap();
let response = send_with_headers(
&ctx.app,
Method::GET,
"/simple/flask/",
vec![("Accept", "application/vnd.pypi.simple.v1+json")],
"",
)
.await;
assert_eq!(response.status(), StatusCode::OK);
let body = body_bytes(response).await;
let json: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert_eq!(json["name"], "flask");
assert!(json["files"].as_array().unwrap().len() == 1);
assert_eq!(json["files"][0]["filename"], "flask-2.0.tar.gz");
assert_eq!(json["files"][0]["digests"]["sha256"], "deadbeef");
}
#[tokio::test]
async fn test_pypi_download_local() {
let ctx = create_test_context();
@@ -607,7 +990,6 @@ mod integration_tests {
async fn test_pypi_not_found_no_proxy() {
let ctx = create_test_context();
// No proxy configured, no local data
let response = send(&ctx.app, Method::GET, "/simple/nonexistent/", "").await;
assert_eq!(response.status(), StatusCode::NOT_FOUND);

View File

@@ -33,8 +33,8 @@ impl ProtectedString {
}
/// Consume and return the inner value
pub fn into_inner(self) -> Zeroizing<String> {
Zeroizing::new(self.inner.clone())
pub fn into_inner(mut self) -> Zeroizing<String> {
Zeroizing::new(std::mem::take(&mut self.inner))
}
/// Check if the secret is empty
@@ -74,8 +74,8 @@ impl From<&str> for ProtectedString {
#[derive(Clone, Zeroize)]
#[zeroize(drop)]
pub struct S3Credentials {
pub access_key_id: String,
#[zeroize(skip)] // access_key_id is not sensitive
pub access_key_id: String,
pub secret_access_key: ProtectedString,
pub region: Option<String>,
}

View File

@@ -102,6 +102,11 @@ fn build_context(
proxy_timeout_zip: 30,
max_zip_size: 10_485_760,
},
cargo: CargoConfig {
proxy: None,
proxy_auth: None,
proxy_timeout: 5,
},
docker: DockerConfig {
proxy_timeout: 5,
upstreams: vec![],

278
scripts/diff-registry.sh Executable file
View File

@@ -0,0 +1,278 @@
#!/usr/bin/env bash
# diff-registry.sh — Differential testing: NORA vs reference registry
# Usage:
# ./scripts/diff-registry.sh docker [nora_url]
# ./scripts/diff-registry.sh npm [nora_url]
# ./scripts/diff-registry.sh cargo [nora_url]
# ./scripts/diff-registry.sh pypi [nora_url]
# ./scripts/diff-registry.sh all [nora_url]
#
# Requires: curl, jq, skopeo (for docker), diff
set -euo pipefail
NORA_URL="${2:-http://localhost:5000}"
TMPDIR=$(mktemp -d)
PASS=0
FAIL=0
SKIP=0
cleanup() { rm -rf "$TMPDIR"; }
trap cleanup EXIT
ok() { PASS=$((PASS+1)); echo " PASS: $1"; }
fail() { FAIL=$((FAIL+1)); echo " FAIL: $1"; }
skip() { SKIP=$((SKIP+1)); echo " SKIP: $1"; }
check_tool() {
if ! command -v "$1" &>/dev/null; then
echo "WARNING: $1 not found, some tests will be skipped"
return 1
fi
return 0
}
# --- Docker ---
diff_docker() {
echo "=== Docker Registry V2 ==="
# 1. /v2/ endpoint returns 200 or 401
local status
status=$(curl -s -o /dev/null -w "%{http_code}" "$NORA_URL/v2/")
if [[ "$status" == "200" || "$status" == "401" ]]; then
ok "/v2/ returns $status"
else
fail "/v2/ returns $status (expected 200 or 401)"
fi
# 2. _catalog returns valid JSON with repositories array
local catalog
catalog=$(curl -s "$NORA_URL/v2/_catalog" 2>/dev/null)
if echo "$catalog" | jq -e '.repositories' &>/dev/null; then
ok "/v2/_catalog has .repositories array"
else
fail "/v2/_catalog invalid JSON: $catalog"
fi
# 3. Push+pull roundtrip with skopeo
if check_tool skopeo; then
local test_image="diff-test/alpine"
local test_tag="diff-$(date +%s)"
# Copy a tiny image to NORA (resolves multi-arch to current platform)
if skopeo copy --dest-tls-verify=false \
docker://docker.io/library/alpine:3.20 \
"docker://${NORA_URL#http*://}/$test_image:$test_tag" 2>/dev/null; then
# Verify manifest structure: must have layers[] and config.digest
skopeo inspect --tls-verify=false --raw \
"docker://${NORA_URL#http*://}/$test_image:$test_tag" \
> "$TMPDIR/nora-manifest.json" 2>/dev/null
local has_layers has_config
has_layers=$(jq -e '.layers | length > 0' "$TMPDIR/nora-manifest.json" 2>/dev/null)
has_config=$(jq -e '.config.digest' "$TMPDIR/nora-manifest.json" 2>/dev/null)
if [[ "$has_layers" == "true" && -n "$has_config" ]]; then
ok "Docker push+pull roundtrip: valid manifest with layers"
else
fail "Docker manifest missing layers or config"
jq . "$TMPDIR/nora-manifest.json" 2>/dev/null || true
fi
# Verify blob is retrievable by digest
local first_layer
first_layer=$(jq -r '.layers[0].digest' "$TMPDIR/nora-manifest.json" 2>/dev/null)
if [[ -n "$first_layer" ]]; then
local blob_status
blob_status=$(curl -s -o /dev/null -w "%{http_code}" \
"$NORA_URL/v2/$test_image/blobs/$first_layer")
if [[ "$blob_status" == "200" ]]; then
ok "Docker blob retrievable by digest"
else
fail "Docker blob GET returned $blob_status"
fi
fi
# Check tags/list
local tags
tags=$(curl -s "$NORA_URL/v2/$test_image/tags/list" 2>/dev/null)
if echo "$tags" | jq -e ".tags[] | select(. == \"$test_tag\")" &>/dev/null; then
ok "tags/list contains pushed tag"
else
fail "tags/list missing pushed tag: $tags"
fi
else
fail "skopeo copy to NORA failed"
fi
else
skip "Docker roundtrip (skopeo not installed)"
fi
}
# --- npm ---
diff_npm() {
echo "=== npm Registry ==="
# 1. Package metadata format
local meta
meta=$(curl -s "$NORA_URL/npm/lodash" 2>/dev/null)
local status=$?
if [[ $status -ne 0 ]]; then
skip "npm metadata (no packages published or upstream unavailable)"
return
fi
if echo "$meta" | jq -e '.name' &>/dev/null; then
ok "npm metadata has .name field"
else
skip "npm metadata (no packages or proxy not configured)"
return
fi
# 2. Tarball URLs point to NORA, not upstream
local tarball_url
tarball_url=$(echo "$meta" | jq -r '.versions | to_entries | last | .value.dist.tarball // empty' 2>/dev/null)
if [[ -n "$tarball_url" ]]; then
if echo "$tarball_url" | grep -qvE "registry.npmjs.org|registry.yarnpkg.com"; then
ok "npm tarball URLs rewritten to NORA"
else
fail "npm tarball URL points to upstream: $tarball_url"
fi
else
skip "npm tarball URL check (no versions)"
fi
}
# --- Cargo ---
diff_cargo() {
echo "=== Cargo Sparse Index ==="
# 1. config.json exists and has dl field
local config
config=$(curl -s "$NORA_URL/cargo/index/config.json" 2>/dev/null)
if echo "$config" | jq -e '.dl' &>/dev/null; then
ok "Cargo config.json has .dl field"
else
fail "Cargo config.json missing .dl: $config"
fi
# 2. config.json has api field
if echo "$config" | jq -e '.api' &>/dev/null; then
ok "Cargo config.json has .api field"
else
fail "Cargo config.json missing .api"
fi
}
# --- PyPI ---
diff_pypi() {
echo "=== PyPI Simple API ==="
# 1. /simple/ returns HTML or JSON
local simple_html
simple_html=$(curl -s -H "Accept: text/html" "$NORA_URL/simple/" 2>/dev/null)
if echo "$simple_html" | grep -qi "<!DOCTYPE\|<html\|simple" &>/dev/null; then
ok "/simple/ returns HTML index"
else
skip "/simple/ HTML (no packages published)"
fi
# 2. PEP 691 JSON response
local simple_json
simple_json=$(curl -s -H "Accept: application/vnd.pypi.simple.v1+json" "$NORA_URL/simple/" 2>/dev/null)
if echo "$simple_json" | jq -e '.projects // .meta' &>/dev/null; then
ok "/simple/ PEP 691 JSON works"
else
skip "/simple/ PEP 691 (not supported or empty)"
fi
}
# --- Go ---
diff_go() {
echo "=== Go Module Proxy ==="
# Basic health: try a known module
local status
status=$(curl -s -o /dev/null -w "%{http_code}" "$NORA_URL/go/golang.org/x/text/@v/list" 2>/dev/null)
if [[ "$status" == "200" || "$status" == "404" ]]; then
ok "Go proxy responds ($status)"
else
skip "Go proxy (status: $status)"
fi
}
# --- Raw ---
diff_raw() {
echo "=== Raw Storage ==="
local test_path="diff-test/test-$(date +%s).txt"
local test_content="diff-registry-test"
# 1. PUT + GET roundtrip
local put_status
put_status=$(curl -s -o /dev/null -w "%{http_code}" -X PUT \
-H "Content-Type: text/plain" \
-d "$test_content" \
"$NORA_URL/raw/$test_path" 2>/dev/null)
if [[ "$put_status" == "200" || "$put_status" == "201" ]]; then
local got
got=$(curl -s "$NORA_URL/raw/$test_path" 2>/dev/null)
if [[ "$got" == "$test_content" ]]; then
ok "Raw PUT+GET roundtrip"
else
fail "Raw GET returned different content: '$got'"
fi
# 2. HEAD returns size
local head_status
head_status=$(curl -s -o /dev/null -w "%{http_code}" -I "$NORA_URL/raw/$test_path" 2>/dev/null)
if [[ "$head_status" == "200" ]]; then
ok "Raw HEAD returns 200"
else
fail "Raw HEAD returned $head_status"
fi
# 3. DELETE
curl -s -o /dev/null -X DELETE "$NORA_URL/raw/$test_path" 2>/dev/null
local after_delete
after_delete=$(curl -s -o /dev/null -w "%{http_code}" "$NORA_URL/raw/$test_path" 2>/dev/null)
if [[ "$after_delete" == "404" ]]; then
ok "Raw DELETE works"
else
fail "Raw DELETE: GET after delete returned $after_delete"
fi
elif [[ "$put_status" == "401" ]]; then
skip "Raw PUT (auth required)"
else
fail "Raw PUT returned $put_status"
fi
}
# --- Main ---
case "${1:-all}" in
docker) diff_docker ;;
npm) diff_npm ;;
cargo) diff_cargo ;;
pypi) diff_pypi ;;
go) diff_go ;;
raw) diff_raw ;;
all)
diff_docker
diff_npm
diff_cargo
diff_pypi
diff_go
diff_raw
;;
*)
echo "Usage: $0 {docker|npm|cargo|pypi|go|raw|all} [nora_url]"
exit 1
;;
esac
echo ""
echo "=== Results: $PASS passed, $FAIL failed, $SKIP skipped ==="
[[ $FAIL -eq 0 ]] && exit 0 || exit 1

View File

@@ -10,12 +10,14 @@ test.describe('NORA Dashboard', () => {
test('dashboard shows registry sections', async ({ page }) => {
await page.goto('/ui/');
// All registry types should be visible
// All 7 registry types should be visible
await expect(page.getByText(/Docker/i).first()).toBeVisible();
await expect(page.getByText(/npm/i).first()).toBeVisible();
await expect(page.getByText(/Maven/i).first()).toBeVisible();
await expect(page.getByText(/PyPI/i).first()).toBeVisible();
await expect(page.getByText(/Cargo/i).first()).toBeVisible();
await expect(page.getByText(/Go/i).first()).toBeVisible();
await expect(page.getByText(/Raw/i).first()).toBeVisible();
});
test('dashboard shows non-zero npm count after proxy fetch', async ({ page, request }) => {
@@ -65,6 +67,8 @@ test.describe('NORA Dashboard', () => {
expect(health.registries.maven).toBe('ok');
expect(health.registries.pypi).toBe('ok');
expect(health.registries.cargo).toBe('ok');
expect(health.registries.go).toBe('ok');
expect(health.registries.raw).toBe('ok');
});
test('OpenAPI docs endpoint accessible', async ({ request }) => {