17 Commits

Author SHA1 Message Date
b3b74b8b2d feat: npm full proxy — URL rewriting, scoped packages, publish, integrity cache (v0.2.31)
npm proxy:
- Rewrite tarball URLs in metadata to point to NORA (was broken — tarballs bypassed NORA)
- Scoped packages (@scope/package) full support in handler and repo index
- Metadata cache TTL (NORA_NPM_METADATA_TTL, default 300s) with stale-while-revalidate
- proxy_auth now wired into fetch_from_proxy (was configured but unused)

npm publish:
- PUT /npm/{package} — accepts standard npm publish payload
- Version immutability — 409 Conflict on duplicate version
- Tarball URL rewriting in published metadata

Security:
- SHA256 integrity verification on cached tarballs (immutable cache)
- Attachment filename validation (path traversal protection)
- Package name mismatch detection (URL vs payload)

Config:
- npm.metadata_ttl — configurable cache TTL (env: NORA_NPM_METADATA_TTL)
2026-03-16 12:32:16 +00:00
d41b55fa3a style: cargo fmt 2026-03-16 08:58:27 +00:00
5a68bfd695 fix: dashboard — docker namespaced repos, npm proxy cache, upstream display (v0.2.30) 2026-03-16 08:55:33 +00:00
9c8fee5a5d docs: rewrite README — new slogan, roadmap, trim TLS/FSTEC, fix config example 2026-03-16 07:39:43 +00:00
bbff337b4c fix: clean up stale smoke test container before run 2026-03-15 22:25:37 +00:00
a73335c549 docs: trim README, link to docs site, fix website URL 2026-03-15 22:21:08 +00:00
ad6aba46b2 chore: remove internal release runbook from public repo 2026-03-15 21:57:05 +00:00
095270d113 fix: smoke test port mapping (4000, not 5000) 2026-03-15 21:54:13 +00:00
769f5fb01d docs: update CHANGELOG and README for v0.2.29 upstream auth 2026-03-15 21:50:14 +00:00
53884e143b v0.2.29: upstream auth, remove dead code, version bump
- Remove unused DockerAuth::fetch_with_auth() method
- Fix basic_auth_header docstring
- Bump to v0.2.29
2026-03-15 21:42:49 +00:00
0eb26f24f7 refactor: extract basic_auth_header helper, add plaintext credential warnings
- basic_auth_header() in config.rs replaces 6 inline STANDARD.encode calls
- warn_plaintext_credentials() logs warning at startup if auth is in config.toml
- All protocol handlers use shared helper instead of duplicating base64 logic
2026-03-15 21:37:51 +00:00
fa962b2d6e feat: upstream auth for all protocols (Docker, Maven, npm, PyPI)
Wire up basic auth credentials for upstream registry proxying:
- Docker: pass configured auth to Bearer token requests
- Maven: support url|auth format in NORA_MAVEN_PROXIES env var
- npm: add NORA_NPM_PROXY_AUTH env var
- PyPI: add NORA_PYPI_PROXY_AUTH env var
- Mask credentials in logs (never log plaintext passwords)

Config examples:
  NORA_DOCKER_UPSTREAMS="https://registry.corp.com|user:pass"
  NORA_MAVEN_PROXIES="https://nexus.corp.com/maven2|user:pass"
  NORA_NPM_PROXY_AUTH="user:pass"
  NORA_PYPI_PROXY_AUTH="user:pass"
2026-03-15 21:29:20 +00:00
a1da4fff1e fix: integration tests match actual protocol support
- Docker, Maven: full push/pull (write support exists)
- npm, PyPI, Cargo: endpoint checks only (read-only proxy, no publish yet)
2026-03-15 19:58:36 +00:00
868c4feca7 feat: add Maven, PyPI, Cargo integration tests
- Maven: PUT artifact, GET and verify checksum
- PyPI: twine upload + pip install
- Cargo: API endpoint check
- Now testing all 5 protocols: Docker, npm, Maven, PyPI, Cargo
2026-03-15 19:53:27 +00:00
5b4cba1392 fix: add npm auth token for integration test publish 2026-03-15 19:49:54 +00:00
ad890be56a feat: add integration tests, release runbook, cache fallback
- CI: integration job — build NORA, docker push/pull, npm publish/install, API checks
- release: cache-from with ignore-error=true (no dependency on localhost:5000)
- RELEASE_RUNBOOK.md: rollback procedure, deploy order, verification steps
2026-03-15 19:36:38 +00:00
3b9ea37b0e fix: cargo fmt, add .gitleaks.toml allowlist for doc examples
- remove extra blank lines in openapi.rs and secrets/mod.rs
- allowlist commit 92155cf (curl -u admin:yourpassword in README)
2026-03-15 19:27:36 +00:00
109 changed files with 1642 additions and 13804 deletions

View File

@@ -1,9 +0,0 @@
FROM rust:1.87-slim@sha256:437507c3e719e4f968033b88d851ffa9f5aceeb2dcc2482cc6cb7647811a55eb
RUN apt-get update && apt-get install -y build-essential pkg-config && rm -rf /var/lib/apt/lists/*
RUN cargo install cargo-fuzz
COPY . /src
WORKDIR /src
RUN cd fuzz && cargo fuzz build 2>/dev/null || true

View File

@@ -1,5 +0,0 @@
language: rust
fuzzing_engines:
- libfuzzer
sanitizers:
- address

142
.githooks/pre-commit Executable file
View File

@@ -0,0 +1,142 @@
#!/bin/bash
# Pre-commit hook to prevent accidental commits of sensitive files
# Enable: git config core.hooksPath .githooks
set -e
RED='\033[0;31m'
YELLOW='\033[1;33m'
GREEN='\033[0;32m'
NC='\033[0m'
# Allowed file extensions (whitelist)
ALLOWED_EXTENSIONS=(
'\.rs$'
'\.toml$'
'\.lock$'
'\.yml$'
'\.yaml$'
'\.json$'
'\.sh$'
'\.html$'
'\.css$'
'\.js$'
'\.gitignore$'
'\.dockerignore$'
'Dockerfile$'
'LICENSE$'
'Makefile$'
)
# Extensions that trigger a warning (not blocked)
WARN_EXTENSIONS=(
'\.md$'
)
# Always blocked patterns (regardless of extension)
BLOCKED_PATTERNS=(
'\.env$'
'\.env\.'
'\.key$'
'\.pem$'
'\.p12$'
'\.pfx$'
'\.htpasswd$'
'secret'
'credential'
'password'
'\.bak$'
'\.swp$'
'\.swo$'
'node_modules/'
'target/debug/'
'\.DS_Store'
)
# Get staged files (only NEW files, not already tracked)
STAGED_FILES=$(git diff --cached --name-only --diff-filter=A)
if [ -z "$STAGED_FILES" ]; then
# No new files, only modifications to existing - allow
exit 0
fi
# Build patterns
ALLOWED_PATTERN=$(IFS='|'; echo "${ALLOWED_EXTENSIONS[*]}")
WARN_PATTERN=$(IFS='|'; echo "${WARN_EXTENSIONS[*]}")
BLOCKED_PATTERN=$(IFS='|'; echo "${BLOCKED_PATTERNS[*]}")
# Check for blocked patterns first
BLOCKED_FILES=$(echo "$STAGED_FILES" | grep -iE "$BLOCKED_PATTERN" || true)
if [ -n "$BLOCKED_FILES" ]; then
echo -e "${RED}BLOCKED: Suspicious files detected in commit${NC}"
echo ""
echo -e "${YELLOW}Files:${NC}"
echo "$BLOCKED_FILES" | sed 's/^/ - /'
echo ""
echo "If intentional, use: git commit --no-verify"
exit 1
fi
# Check for files with unknown extensions
UNKNOWN_FILES=""
WARN_FILES=""
while IFS= read -r file; do
[ -z "$file" ] && continue
if echo "$file" | grep -qE "$BLOCKED_PATTERN"; then
continue # Already handled above
elif echo "$file" | grep -qE "$WARN_PATTERN"; then
WARN_FILES="$WARN_FILES$file"$'\n'
elif ! echo "$file" | grep -qE "$ALLOWED_PATTERN"; then
UNKNOWN_FILES="$UNKNOWN_FILES$file"$'\n'
fi
done <<< "$STAGED_FILES"
# Warn about .md files
if [ -n "$WARN_FILES" ]; then
echo -e "${YELLOW}WARNING: Markdown files in commit:${NC}"
echo "$WARN_FILES" | sed '/^$/d' | sed 's/^/ - /'
echo ""
fi
# Block unknown extensions
if [ -n "$UNKNOWN_FILES" ]; then
echo -e "${RED}BLOCKED: Files with unknown extensions:${NC}"
echo "$UNKNOWN_FILES" | sed '/^$/d' | sed 's/^/ - /'
echo ""
echo "Allowed extensions: rs, toml, lock, yml, yaml, json, sh, html, css, js, md"
echo "If intentional, use: git commit --no-verify"
exit 1
fi
# Check for large files (>5MB)
LARGE_FILES=$(echo "$STAGED_FILES" | while read f; do
if [ -f "$f" ]; then
size=$(stat -f%z "$f" 2>/dev/null || stat -c%s "$f" 2>/dev/null || echo 0)
if [ "$size" -gt 5242880 ]; then
echo "$f ($(numfmt --to=iec $size 2>/dev/null || echo "${size}B"))"
fi
fi
done)
if [ -n "$LARGE_FILES" ]; then
echo -e "${YELLOW}WARNING: Large files (>5MB) in commit:${NC}"
echo "$LARGE_FILES" | sed 's/^/ - /'
echo ""
fi
# Run cargo fmt check if Rust files changed
if git diff --cached --name-only | grep -q '\.rs$'; then
if command -v cargo &> /dev/null; then
if ! cargo fmt --check &> /dev/null; then
echo -e "${RED}BLOCKED: cargo fmt check failed${NC}"
echo "Run: cargo fmt"
exit 1
fi
fi
fi
exit 0

2
.github/CODEOWNERS vendored
View File

@@ -1,2 +0,0 @@
# Default owner for everything
* @devitway

View File

@@ -1,57 +0,0 @@
name: Bug Report
description: Report a bug in NORA
labels: [bug]
body:
- type: textarea
id: description
attributes:
label: Description
description: What happened?
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected Behavior
description: What did you expect to happen?
- type: textarea
id: reproduce
attributes:
label: Steps to Reproduce
description: How can we reproduce this?
validations:
required: true
- type: input
id: version
attributes:
label: NORA Version
description: Output of 'nora --version' or Docker tag
placeholder: v0.3.0
validations:
required: true
- type: dropdown
id: registry
attributes:
label: Registry Protocol
options:
- Docker/OCI
- npm
- Maven
- PyPI
- Cargo
- Go
- Raw
- UI/Dashboard
- Other
- type: dropdown
id: storage
attributes:
label: Storage Backend
options:
- Local filesystem
- S3-compatible
- type: textarea
id: logs
attributes:
label: Relevant Logs
render: shell

View File

@@ -1,37 +0,0 @@
name: Feature Request
description: Suggest a new feature for NORA
labels: [enhancement]
body:
- type: textarea
id: problem
attributes:
label: Problem
description: What problem does this solve?
validations:
required: true
- type: textarea
id: solution
attributes:
label: Proposed Solution
description: How would you like it to work?
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives Considered
description: Other approaches you've thought about
- type: dropdown
id: registry
attributes:
label: Related Registry
options:
- Docker/OCI
- npm
- Maven
- PyPI
- Cargo
- Go
- Raw
- UI/Dashboard
- Core/General

View File

@@ -1,16 +0,0 @@
## Summary
<!-- What does this PR do? -->
## Changes
<!-- List key changes -->
## Checklist
- [ ] passes
- [ ] passes
- [ ] passes
- [ ] No in production code
- [ ] New public API has documentation
- [ ] CHANGELOG updated (if user-facing change)

View File

@@ -1,3 +0,0 @@
self-hosted-runner:
labels:
- nora

Binary file not shown.

Before

Width:  |  Height:  |  Size: 279 KiB

View File

@@ -1,15 +0,0 @@
## What does this PR do?
<!-- Brief description of the change -->
## Related issue
<!-- Link to issue, e.g. Fixes #123 -->
## Checklist
- [ ] `cargo fmt` passes
- [ ] `cargo clippy` passes with no warnings
- [ ] `cargo test --lib --bin nora` passes
- [ ] New functionality includes tests
- [ ] CHANGELOG.md updated (if user-facing change)

View File

@@ -6,20 +6,18 @@ on:
pull_request: pull_request:
branches: [main] branches: [main]
permissions: read-all
jobs: jobs:
test: test:
name: Test name: Test
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - uses: actions/checkout@v6
- name: Install Rust - name: Install Rust
uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable uses: dtolnay/rust-toolchain@stable
- name: Cache cargo - name: Cache cargo
uses: Swatinem/rust-cache@42dc69e1aa15d09112580998cf2ef0119e2e91ae # v2 uses: Swatinem/rust-cache@v2
- name: Check formatting - name: Check formatting
run: cargo fmt --check run: cargo fmt --check
@@ -30,116 +28,41 @@ jobs:
- name: Run tests - name: Run tests
run: cargo test --package nora-registry run: cargo test --package nora-registry
lint-workflows:
name: Lint Workflows
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Install actionlint
run: |
ACTIONLINT_VERSION=1.7.12
curl -sLO "https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz"
tar xzf "actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" actionlint
- name: Run actionlint
run: ./actionlint -ignore "shellcheck reported issue" -ignore "SC[0-9]"
coverage:
name: Coverage
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
permissions:
contents: read
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Install Rust
uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable
- name: Cache cargo
uses: Swatinem/rust-cache@42dc69e1aa15d09112580998cf2ef0119e2e91ae # v2
- name: Install tarpaulin
run: cargo install cargo-tarpaulin --locked
- name: Run coverage
run: |
cargo tarpaulin --config tarpaulin.toml 2>&1 | tee /tmp/tarpaulin.log
COVERAGE=$(python3 -c "import json; d=json.load(open('coverage/tarpaulin-report.json')); print(f\"{d['coverage']:.1f}\")")
echo "COVERAGE=$COVERAGE" >> $GITHUB_ENV
echo "Coverage: $COVERAGE%"
- name: Update coverage badge
uses: schneegans/dynamic-badges-action@0e50b8bad39e7e1afd3e4e9c2b7dd145fad07501 # v1.8.0
with:
auth: ${{ secrets.GIST_TOKEN }}
gistID: ${{ vars.COVERAGE_GIST_ID }}
filename: nora-coverage.json
label: coverage
message: ${{ env.COVERAGE }}%
valColorRange: ${{ env.COVERAGE }}
minColorRange: 0
maxColorRange: 100
security: security:
name: Security name: Security
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions: permissions:
contents: read contents: read
security-events: write security-events: write # for uploading SARIF to GitHub Security tab
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - uses: actions/checkout@v6
with: with:
fetch-depth: 0 fetch-depth: 0 # full history required for gitleaks
- name: Install Rust - name: Install Rust
uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable uses: dtolnay/rust-toolchain@stable
- name: Cache cargo - name: Cache cargo
uses: Swatinem/rust-cache@42dc69e1aa15d09112580998cf2ef0119e2e91ae # v2 uses: Swatinem/rust-cache@v2
# ── Secrets ──────────────────────────────────────────────────────────── # ── Secrets ────────────────────────────────────────────────────────────
- name: Gitleaks — scan for hardcoded secrets - name: Gitleaks — scan for hardcoded secrets
run: | run: |
curl -sL https://github.com/gitleaks/gitleaks/releases/download/v8.21.2/gitleaks_8.21.2_linux_x64.tar.gz \ curl -sL https://github.com/gitleaks/gitleaks/releases/download/v8.21.2/gitleaks_8.21.2_linux_x64.tar.gz \
| tar xz -C /usr/local/bin gitleaks | tar xz -C /usr/local/bin gitleaks
gitleaks detect --source . --config .gitleaks.toml --exit-code 1 --report-format sarif --report-path gitleaks.sarif gitleaks detect --source . --exit-code 1 --report-format sarif --report-path gitleaks.sarif
# ── CVE in Rust dependencies ──────────────────────────────────────────── # ── CVE in Rust dependencies ────────────────────────────────────────────
- name: Install cargo-audit - name: Install cargo-audit
run: cargo install cargo-audit --locked run: cargo install cargo-audit --locked
- name: cargo audit — RustSec advisory database - name: cargo audit — RustSec advisory database
run: | run: cargo audit --ignore RUSTSEC-2025-0119 # known: number_prefix via indicatif
cargo audit --ignore RUSTSEC-2025-0119
cargo audit --ignore RUSTSEC-2025-0119 --json > /tmp/audit.json || true
- name: Upload cargo-audit results as SARIF
if: always()
run: |
python3 -c "
import json, sys
sarif = {
'version': '2.1.0',
'\$schema': 'https://raw.githubusercontent.com/oasis-tcs/sarif-spec/main/sarif-2.1/schema/sarif-schema-2.1.0.json',
'runs': [{'tool': {'driver': {'name': 'cargo-audit', 'version': '0.21', 'informationUri': 'https://github.com/rustsec/rustsec'}}, 'results': []}]
}
with open('cargo-audit.sarif', 'w') as f:
json.dump(sarif, f)
"
- name: Upload SAST results to GitHub Security tab
uses: github/codeql-action/upload-sarif@a60c4df7a135c7317c1e9ddf9b5a9b07a910dda9 # v4
if: always()
with:
sarif_file: cargo-audit.sarif
category: cargo-audit
# ── Licenses, banned crates, supply chain policy ──────────────────────── # ── Licenses, banned crates, supply chain policy ────────────────────────
- name: cargo deny — licenses and banned crates - name: cargo deny — licenses and banned crates
uses: EmbarkStudios/cargo-deny-action@82eb9f621fbc699dd0918f3ea06864c14cc84246 # v2 uses: EmbarkStudios/cargo-deny-action@v2
with: with:
command: check command: check
arguments: --all-features arguments: --all-features
@@ -147,17 +70,17 @@ jobs:
# ── CVE scan of source tree and Cargo.lock ────────────────────────────── # ── CVE scan of source tree and Cargo.lock ──────────────────────────────
- name: Trivy — filesystem scan (Cargo.lock + source) - name: Trivy — filesystem scan (Cargo.lock + source)
if: always() if: always()
uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1 # 0.35.0 uses: aquasecurity/trivy-action@0.35.0
with: with:
scan-type: fs scan-type: fs
scan-ref: . scan-ref: .
format: sarif format: sarif
output: trivy-fs.sarif output: trivy-fs.sarif
severity: HIGH,CRITICAL severity: HIGH,CRITICAL
exit-code: 1 exit-code: 1 # block pipeline on HIGH/CRITICAL vulnerabilities
- name: Upload Trivy fs results to GitHub Security tab - name: Upload Trivy fs results to GitHub Security tab
uses: github/codeql-action/upload-sarif@a60c4df7a135c7317c1e9ddf9b5a9b07a910dda9 # v4 uses: github/codeql-action/upload-sarif@v4
if: always() if: always()
with: with:
sarif_file: trivy-fs.sarif sarif_file: trivy-fs.sarif
@@ -169,17 +92,18 @@ jobs:
needs: test needs: test
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - uses: actions/checkout@v6
- name: Install Rust - name: Install Rust
uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7 # stable uses: dtolnay/rust-toolchain@stable
- name: Cache cargo - name: Cache cargo
uses: Swatinem/rust-cache@42dc69e1aa15d09112580998cf2ef0119e2e91ae # v2 uses: Swatinem/rust-cache@v2
- name: Build NORA - name: Build NORA
run: cargo build --release --package nora-registry run: cargo build --release --package nora-registry
# -- Start NORA --
- name: Start NORA - name: Start NORA
run: | run: |
NORA_STORAGE_PATH=/tmp/nora-data ./target/release/nora & NORA_STORAGE_PATH=/tmp/nora-data ./target/release/nora &
@@ -188,6 +112,7 @@ jobs:
done done
curl -sf http://localhost:4000/health | jq . curl -sf http://localhost:4000/health | jq .
# -- Docker push/pull --
- name: Configure Docker for insecure registry - name: Configure Docker for insecure registry
run: | run: |
echo '{"insecure-registries": ["localhost:4000"]}' | sudo tee /etc/docker/daemon.json echo '{"insecure-registries": ["localhost:4000"]}' | sudo tee /etc/docker/daemon.json
@@ -208,35 +133,38 @@ jobs:
curl -sf http://localhost:4000/v2/_catalog | jq . curl -sf http://localhost:4000/v2/_catalog | jq .
curl -sf http://localhost:4000/v2/test/alpine/tags/list | jq . curl -sf http://localhost:4000/v2/test/alpine/tags/list | jq .
# -- npm (read-only proxy, no publish support yet) --
- name: npm — verify registry endpoint - name: npm — verify registry endpoint
run: | run: |
STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4000/npm/lodash) STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4000/npm/lodash)
echo "npm endpoint returned: $STATUS" echo "npm endpoint returned: $STATUS"
[ "$STATUS" != "000" ] && echo "npm endpoint OK" || (echo "npm endpoint unreachable" && exit 1) [ "$STATUS" != "000" ] && echo "npm endpoint OK" || (echo "npm endpoint unreachable" && exit 1)
# -- Maven deploy/download --
- name: Maven — deploy and download artifact - name: Maven — deploy and download artifact
run: | run: |
echo "test-artifact-content-$(date +%s)" > /tmp/test-artifact.jar echo "test-artifact-content-$(date +%s)" > /tmp/test-artifact.jar
CHECKSUM=$(sha256sum /tmp/test-artifact.jar | cut -d' ' -f1) CHECKSUM=$(sha256sum /tmp/test-artifact.jar | cut -d' ' -f1)
curl -sf -X PUT --data-binary @/tmp/test-artifact.jar \ curl -sf -X PUT --data-binary @/tmp/test-artifact.jar http://localhost:4000/maven2/com/example/test-lib/1.0.0/test-lib-1.0.0.jar
http://localhost:4000/maven2/com/example/test-lib/1.0.0/test-lib-1.0.0.jar curl -sf -o /tmp/downloaded.jar http://localhost:4000/maven2/com/example/test-lib/1.0.0/test-lib-1.0.0.jar
curl -sf -o /tmp/downloaded.jar \
http://localhost:4000/maven2/com/example/test-lib/1.0.0/test-lib-1.0.0.jar
DOWNLOAD_CHECKSUM=$(sha256sum /tmp/downloaded.jar | cut -d' ' -f1) DOWNLOAD_CHECKSUM=$(sha256sum /tmp/downloaded.jar | cut -d' ' -f1)
[ "$CHECKSUM" = "$DOWNLOAD_CHECKSUM" ] && echo "Maven deploy/download OK" || (echo "Checksum mismatch!" && exit 1) [ "$CHECKSUM" = "$DOWNLOAD_CHECKSUM" ] && echo "Maven deploy/download OK" || (echo "Checksum mismatch!" && exit 1)
# -- PyPI (read-only proxy, no upload support yet) --
- name: PyPI — verify simple index - name: PyPI — verify simple index
run: | run: |
STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4000/simple/) STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4000/simple/)
echo "PyPI simple index returned: $STATUS" echo "PyPI simple index returned: $STATUS"
[ "$STATUS" = "200" ] && echo "PyPI endpoint OK" || (echo "Expected 200, got $STATUS" && exit 1) [ "$STATUS" = "200" ] && echo "PyPI endpoint OK" || (echo "Expected 200, got $STATUS" && exit 1)
# -- Cargo (read-only proxy, no publish support yet) --
- name: Cargo — verify registry API responds - name: Cargo — verify registry API responds
run: | run: |
STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4000/cargo/api/v1/crates/serde) STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4000/cargo/api/v1/crates/serde)
echo "Cargo API returned: $STATUS" echo "Cargo API returned: $STATUS"
[ "$STATUS" != "000" ] && echo "Cargo endpoint OK" || (echo "Cargo endpoint unreachable" && exit 1) [ "$STATUS" != "000" ] && echo "Cargo endpoint OK" || (echo "Cargo endpoint unreachable" && exit 1)
# -- API checks --
- name: API — health, ready, metrics - name: API — health, ready, metrics
run: | run: |
curl -sf http://localhost:4000/health | jq .status curl -sf http://localhost:4000/health | jq .status

View File

@@ -1,36 +0,0 @@
name: CodeQL
on:
push:
branches: [main]
pull_request:
branches: [main]
schedule:
- cron: '0 6 * * 1' # Weekly Monday 06:00 UTC
permissions: read-all
jobs:
analyze:
name: CodeQL Analysis
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Initialize CodeQL
uses: github/codeql-action/init@a60c4df7a135c7317c1e9ddf9b5a9b07a910dda9 # v4
with:
languages: actions
queries: security-and-quality
- name: Autobuild
uses: github/codeql-action/autobuild@a60c4df7a135c7317c1e9ddf9b5a9b07a910dda9 # v4
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@a60c4df7a135c7317c1e9ddf9b5a9b07a910dda9 # v4
with:
category: codeql

View File

@@ -4,8 +4,6 @@ on:
push: push:
tags: ['v*'] tags: ['v*']
permissions: read-all
env: env:
REGISTRY: ghcr.io REGISTRY: ghcr.io
NORA: localhost:5000 NORA: localhost:5000
@@ -15,15 +13,12 @@ jobs:
build: build:
name: Build & Push name: Build & Push
runs-on: [self-hosted, nora] runs-on: [self-hosted, nora]
outputs:
hash: ${{ steps.hash.outputs.hash }}
permissions: permissions:
contents: read contents: read
packages: write packages: write
id-token: write # Sigstore cosign keyless signing
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - uses: actions/checkout@v6
- name: Set up Rust - name: Set up Rust
run: | run: |
@@ -37,26 +32,19 @@ jobs:
cp target/x86_64-unknown-linux-musl/release/nora ./nora cp target/x86_64-unknown-linux-musl/release/nora ./nora
- name: Upload binary artifact - name: Upload binary artifact
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7 uses: actions/upload-artifact@v7
with: with:
name: nora-binary-${{ github.run_id }} name: nora-binary-${{ github.run_id }}
path: ./nora path: ./nora
retention-days: 1 retention-days: 1
- name: Compute binary hash for SLSA provenance
id: hash
run: |
cp target/x86_64-unknown-linux-musl/release/nora ./nora-linux-amd64
sha256sum nora-linux-amd64 | base64 -w0 > hash.txt
echo "hash=$(cat hash.txt)" >> $GITHUB_OUTPUT
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4 uses: docker/setup-buildx-action@v4
with: with:
driver-opts: network=host driver-opts: network=host
- name: Log in to GitHub Container Registry - name: Log in to GitHub Container Registry
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4 uses: docker/login-action@v4
with: with:
registry: ${{ env.REGISTRY }} registry: ${{ env.REGISTRY }}
username: ${{ github.actor }} username: ${{ github.actor }}
@@ -65,9 +53,10 @@ jobs:
# ── Alpine ─────────────────────────────────────────────────────────────── # ── Alpine ───────────────────────────────────────────────────────────────
- name: Extract metadata (alpine) - name: Extract metadata (alpine)
id: meta-alpine id: meta-alpine
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6 uses: docker/metadata-action@v6
with: with:
images: | images: |
${{ env.NORA }}/${{ env.IMAGE_NAME }}
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: | tags: |
type=semver,pattern={{version}} type=semver,pattern={{version}}
@@ -75,7 +64,7 @@ jobs:
type=raw,value=latest type=raw,value=latest
- name: Build and push (alpine) - name: Build and push (alpine)
uses: docker/build-push-action@d08e5c354a6adb9ed34480a06d141179aa583294 # v7 uses: docker/build-push-action@v7
with: with:
context: . context: .
file: Dockerfile file: Dockerfile
@@ -83,22 +72,25 @@ jobs:
push: true push: true
tags: ${{ steps.meta-alpine.outputs.tags }} tags: ${{ steps.meta-alpine.outputs.tags }}
labels: ${{ steps.meta-alpine.outputs.labels }} labels: ${{ steps.meta-alpine.outputs.labels }}
cache-from: type=registry,ref=${{ env.NORA }}/${{ env.IMAGE_NAME }}-cache:alpine,ignore-error=true
cache-to: type=registry,ref=${{ env.NORA }}/${{ env.IMAGE_NAME }}-cache:alpine,mode=max
# ── RED OS ─────────────────────────────────────────────────────────────── # ── RED OS ───────────────────────────────────────────────────────────────
- name: Extract metadata (redos) - name: Extract metadata (redos)
id: meta-redos id: meta-redos
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6 uses: docker/metadata-action@v6
with: with:
images: | images: |
${{ env.NORA }}/${{ env.IMAGE_NAME }}
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
flavor: suffix=-redos,onlatest=true flavor: suffix=-redos,onlatest=true
tags: | tags: |
type=semver,pattern={{version}} type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}} type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest type=raw,value=redos
- name: Build and push (redos) - name: Build and push (redos)
uses: docker/build-push-action@d08e5c354a6adb9ed34480a06d141179aa583294 # v7 uses: docker/build-push-action@v7
with: with:
context: . context: .
file: Dockerfile.redos file: Dockerfile.redos
@@ -106,22 +98,25 @@ jobs:
push: true push: true
tags: ${{ steps.meta-redos.outputs.tags }} tags: ${{ steps.meta-redos.outputs.tags }}
labels: ${{ steps.meta-redos.outputs.labels }} labels: ${{ steps.meta-redos.outputs.labels }}
cache-from: type=registry,ref=${{ env.NORA }}/${{ env.IMAGE_NAME }}-cache:redos,ignore-error=true
cache-to: type=registry,ref=${{ env.NORA }}/${{ env.IMAGE_NAME }}-cache:redos,mode=max
# ── Astra Linux SE ─────────────────────────────────────────────────────── # ── Astra Linux SE ───────────────────────────────────────────────────────
- name: Extract metadata (astra) - name: Extract metadata (astra)
id: meta-astra id: meta-astra
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6 uses: docker/metadata-action@v6
with: with:
images: | images: |
${{ env.NORA }}/${{ env.IMAGE_NAME }}
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
flavor: suffix=-astra,onlatest=true flavor: suffix=-astra,onlatest=true
tags: | tags: |
type=semver,pattern={{version}} type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}} type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest type=raw,value=astra
- name: Build and push (astra) - name: Build and push (astra)
uses: docker/build-push-action@d08e5c354a6adb9ed34480a06d141179aa583294 # v7 uses: docker/build-push-action@v7
with: with:
context: . context: .
file: Dockerfile.astra file: Dockerfile.astra
@@ -129,23 +124,15 @@ jobs:
push: true push: true
tags: ${{ steps.meta-astra.outputs.tags }} tags: ${{ steps.meta-astra.outputs.tags }}
labels: ${{ steps.meta-astra.outputs.labels }} labels: ${{ steps.meta-astra.outputs.labels }}
cache-from: type=registry,ref=${{ env.NORA }}/${{ env.IMAGE_NAME }}-cache:astra,ignore-error=true
cache-to: type=registry,ref=${{ env.NORA }}/${{ env.IMAGE_NAME }}-cache:astra,mode=max
# ── Smoke test ────────────────────────────────────────────────────────── # ── Smoke test ──────────────────────────────────────────────────────────
- name: Install cosign
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v3
- name: Sign Docker images (keyless Sigstore)
run: |
TAGS=($(echo "${{ steps.meta-alpine.outputs.tags }}" | tr "\n" " "))
for tag in "${TAGS[@]}"; do
[[ "$tag" == *"localhost"* ]] && continue
cosign sign --yes "$tag"
done
- name: Smoke test — verify alpine image starts and responds - name: Smoke test — verify alpine image starts and responds
run: | run: |
docker rm -f nora-smoke 2>/dev/null || true docker rm -f nora-smoke 2>/dev/null || true
docker run --rm -d --name nora-smoke -p 5555:4000 -e NORA_HOST=0.0.0.0 ghcr.io/${{ github.repository }}:${{ steps.meta-alpine.outputs.version }} docker run --rm -d --name nora-smoke -p 5555:4000 -e NORA_HOST=0.0.0.0 \
${{ env.NORA }}/${{ env.IMAGE_NAME }}:latest
for i in $(seq 1 10); do for i in $(seq 1 10); do
curl -sf http://localhost:5555/health && break || sleep 2 curl -sf http://localhost:5555/health && break || sleep 2
done done
@@ -178,52 +165,39 @@ jobs:
run: echo "tag=${GITHUB_REF_NAME#v}" >> $GITHUB_OUTPUT run: echo "tag=${GITHUB_REF_NAME#v}" >> $GITHUB_OUTPUT
- name: Trivy — image scan (${{ matrix.name }}) - name: Trivy — image scan (${{ matrix.name }})
uses: aquasecurity/trivy-action@57a97c7e7821a5776cebc9bb87c984fa69cba8f1 # 0.35.0 uses: aquasecurity/trivy-action@0.35.0
with: with:
scan-type: image scan-type: image
image-ref: ghcr.io/${{ github.repository }}:${{ steps.ver.outputs.tag }}${{ matrix.suffix }} image-ref: ${{ env.NORA }}/${{ env.IMAGE_NAME }}:${{ steps.ver.outputs.tag }}${{ matrix.suffix }}
format: sarif format: sarif
output: trivy-image-${{ matrix.name }}.sarif output: trivy-image-${{ matrix.name }}.sarif
severity: HIGH,CRITICAL severity: HIGH,CRITICAL
exit-code: 0 exit-code: 1
- name: Upload Trivy image results to GitHub Security tab - name: Upload Trivy image results to GitHub Security tab
uses: github/codeql-action/upload-sarif@a60c4df7a135c7317c1e9ddf9b5a9b07a910dda9 # v4 uses: github/codeql-action/upload-sarif@v4
if: always() if: always()
with: with:
sarif_file: trivy-image-${{ matrix.name }}.sarif sarif_file: trivy-image-${{ matrix.name }}.sarif
category: trivy-image-${{ matrix.name }} category: trivy-image-${{ matrix.name }}
provenance:
name: SLSA Provenance
needs: build
permissions:
actions: read
id-token: write
contents: write
uses: slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@v2.1.0
with:
base64-subjects: "${{ needs.build.outputs.hash }}"
upload-assets: true
release: release:
name: GitHub Release name: GitHub Release
runs-on: [self-hosted, nora] runs-on: [self-hosted, nora]
needs: [build, scan, provenance] needs: [build, scan]
permissions: permissions:
contents: write contents: write
id-token: write # Sigstore cosign keyless signing packages: read
packages: write # cosign needs push for signatures
steps: steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 - uses: actions/checkout@v6
- name: Set version tag (strip leading v) - name: Set version tag (strip leading v)
id: ver id: ver
run: echo "tag=${GITHUB_REF_NAME#v}" >> $GITHUB_OUTPUT run: echo "tag=${GITHUB_REF_NAME#v}" >> $GITHUB_OUTPUT
- name: Download binary artifact - name: Download binary artifact
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4 uses: actions/download-artifact@v4
with: with:
name: nora-binary-${{ github.run_id }} name: nora-binary-${{ github.run_id }}
path: ./artifacts path: ./artifacts
@@ -237,38 +211,26 @@ jobs:
cat nora-linux-amd64.sha256 cat nora-linux-amd64.sha256
- name: Generate SBOM (SPDX) - name: Generate SBOM (SPDX)
uses: anchore/sbom-action@57aae528053a48a3f6235f2d9461b05fbcb7366d # v0 uses: anchore/sbom-action@v0
with: with:
image: ${{ env.NORA }}/${{ env.IMAGE_NAME }}:${{ steps.ver.outputs.tag }}
format: spdx-json format: spdx-json
output-file: nora-${{ github.ref_name }}.sbom.spdx.json output-file: nora-${{ github.ref_name }}.sbom.spdx.json
- name: Generate SBOM (CycloneDX) - name: Generate SBOM (CycloneDX)
uses: anchore/sbom-action@57aae528053a48a3f6235f2d9461b05fbcb7366d # v0 uses: anchore/sbom-action@v0
with: with:
image: ${{ env.NORA }}/${{ env.IMAGE_NAME }}:${{ steps.ver.outputs.tag }}
format: cyclonedx-json format: cyclonedx-json
output-file: nora-${{ github.ref_name }}.sbom.cdx.json output-file: nora-${{ github.ref_name }}.sbom.cdx.json
- name: Install cosign
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v3
- name: Sign binary with cosign (keyless Sigstore)
run: |
cosign sign-blob --yes \
--output-signature nora-linux-amd64.sig \
--output-certificate nora-linux-amd64.cert \
--bundle nora-linux-amd64.bundle \
./nora-linux-amd64
- name: Create Release - name: Create Release
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2 uses: softprops/action-gh-release@v2
with: with:
generate_release_notes: true generate_release_notes: true
files: | files: |
nora-linux-amd64 nora-linux-amd64
nora-linux-amd64.sha256 nora-linux-amd64.sha256
nora-linux-amd64.sig
nora-linux-amd64.cert
nora-linux-amd64.bundle
nora-${{ github.ref_name }}.sbom.spdx.json nora-${{ github.ref_name }}.sbom.spdx.json
nora-${{ github.ref_name }}.sbom.cdx.json nora-${{ github.ref_name }}.sbom.cdx.json
body: | body: |
@@ -290,17 +252,17 @@ jobs:
**Alpine (standard):** **Alpine (standard):**
```bash ```bash
docker pull ghcr.io/${{ github.repository }}:${{ steps.ver.outputs.tag }} docker pull ghcr.io/${{ github.repository }}:${{ github.ref_name }}
``` ```
**RED OS:** **RED OS:**
```bash ```bash
docker pull ghcr.io/${{ github.repository }}:${{ steps.ver.outputs.tag }}-redos docker pull ghcr.io/${{ github.repository }}:${{ github.ref_name }}-redos
``` ```
**Astra Linux SE:** **Astra Linux SE:**
```bash ```bash
docker pull ghcr.io/${{ github.repository }}:${{ steps.ver.outputs.tag }}-astra docker pull ghcr.io/${{ github.repository }}:${{ github.ref_name }}-astra
``` ```
## Changelog ## Changelog

View File

@@ -1,38 +0,0 @@
name: OpenSSF Scorecard
on:
push:
branches: [main]
schedule:
- cron: '0 6 * * 1'
permissions: read-all
jobs:
analysis:
name: Scorecard analysis
runs-on: ubuntu-latest
permissions:
security-events: write
id-token: write
contents: read
actions: read
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
persist-credentials: false
- name: Run OpenSSF Scorecard
uses: ossf/scorecard-action@4eaacf0543bb3f2c246792bd56e8cdeffafb205a # v2.4.3
with:
results_file: results.sarif
results_format: sarif
publish_results: true
repo_token: ${{ secrets.SCORECARD_TOKEN || secrets.GITHUB_TOKEN }}
- name: Upload Scorecard results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v4 # tag required by scorecard webapp verification
with:
sarif_file: results.sarif
category: scorecard

27
.gitignore vendored
View File

@@ -4,24 +4,19 @@ data/
.env .env
.env.* .env.*
*.log *.log
internal config
# Backup files # Backup files
*.bak *.bak
# Generated by CI # Internal files
*.cdx.json SESSION*.md
# Playwright / Node
node_modules/
package-lock.json
/tmp/
# Working files (never commit)
SESSION_*.md
TODO.md TODO.md
FEEDBACK.txt ROADMAP*.md
*.session.txt docs-site/
*-this-session-*.txt docs/
nora-review.sh *.txt
coverage/
target/criterion/ ## Internal files
.internal/
examples/

View File

@@ -1,13 +1,8 @@
# Gitleaks configuration # Gitleaks configuration
# https://github.com/gitleaks/gitleaks # https://github.com/gitleaks/gitleaks
title = "NORA gitleaks rules"
[allowlist] [allowlist]
description = "Global allowlist for false positives" description = "Allowlist for false positives"
paths = [
'''\.gitleaks\.toml$''', # Documentation examples with placeholder credentials
] commits = ["92155cf6574d89f93ee68503a7b68455ceaa19af"]
regexTarget = "match"
# Test placeholder tokens (e.g. nra_00112233...)
regexes = ['''nra_0{2}[0-9a-f]{30}''']

View File

@@ -1,227 +1,4 @@
# Changelog # Changelog
## [Unreleased]
## [0.5.0] - 2026-04-07
### Added
- **Cargo sparse index (RFC 2789)** — cargo can now use NORA as a proper registry with `sparse+http://` protocol, including `config.json`, prefix-based index lookup, and `cargo publish` wire format support
- **Cargo publish** — full publish flow with wire format parsing, version immutability (409 Conflict), SHA-256 checksums in sparse index, and proper `warnings` response format
- **PyPI twine upload** — `twine upload` via multipart/form-data with SHA-256 verification, filename validation, and version immutability
- **PEP 691 JSON API** — content negotiation via `Accept: application/vnd.pypi.simple.v1+json` for package index and version listing, with hash digests in responses
- 577 total tests (up from 504), including 25 new Cargo tests and 18 new PyPI tests
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Cargo dependency field mapping: `version_req` correctly renamed to `req` and `explicit_name_in_toml` to `package` in sparse index entries, matching Cargo registry specification
- Cargo crate names normalized to lowercase across all endpoints (publish, download, metadata, sparse index) for consistent storage keys
- Cargo publish write ordering: index written before .crate tarball to prevent orphaned files on partial failure
- Cargo conflict errors now return Cargo-compatible JSON format (`{"errors": [{"detail": "..."}]}`)
- PyPI hash fragments preserved when rewriting upstream links (PEP 503 compliance)
- Redundant path traversal checks removed from crate name validation (charset already excludes unsafe characters)
### Changed
- Cargo sparse index and config.json responses include `Cache-Control: public, max-age=300`
- Cargo .crate downloads include `Cache-Control: public, max-age=31536000, immutable` and `Content-Type: application/x-tar`
- axum upgraded with `multipart` feature for PyPI upload support
## [0.4.0] - 2026-04-05
### Added
- **Docker image mirroring** — nora mirror docker fetches manifests and blobs from upstream registries (Docker Hub, ghcr.io, etc.) and pushes into NORA (#41)
- **yarn.lock support** — nora mirror yarn parses v1 format with scoped packages and dedup (#44)
- **--json output for mirror** — nora mirror npm --json outputs structured JSON for CI/CD pipelines (#43)
- **Storage size in /health** — total_size_bytes field in health endpoint response (#42)
- 499 total tests (up from 466), 61.5% code coverage (up from 43%)
### Changed
- fetch_blob_from_upstream and fetch_manifest_from_upstream are now pub for reuse in mirror module
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- tarpaulin exclude-files paths corrected to workspace-relative (coverage jumped from 29% to 61%) (#92)
- Env var naming unified across all registries (#39, #90)
## [0.3.1] - 2026-04-05
### Added
- **Token verification cache** — in-memory with 5min TTL, eliminates repeated Argon2id on every request
- **Property-based tests** (proptest) for Docker/OCI manifest parsers (#84)
- 466 total tests, 43% code coverage (up from 22%) (#87)
- MSRV declared in Cargo.toml (#84)
### Changed
- Upload sessions moved from global static to AppState
- Blocking I/O replaced with async in hot paths
- Production docker-compose includes Caddy reverse proxy
- clippy.toml added for consistent lint rules
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Proxy request deduplication — concurrent requests coalesced (#83)
- Multi-registry GC now handles all 7 registry types (#83)
- TOCTOU race condition in credential validation (#83)
- Config validation at startup — fail fast with clear errors (#73)
- Raw registry in dashboard sidebar, footer stats updated (#64)
- tarpaulin.toml config format (#88)
### Security
- sha2 0.10→0.11, hmac 0.12→0.13 (#75)
- Credential hygiene — cleared from memory after use (#83)
- cosign-installer 3.8.0→4.1.1 (#71)
### Documentation
- Development Setup in CONTRIBUTING.md (#76)
- Roadmap consolidated into README (#65, #66)
- Helm OCI docs and logging env vars documented
## [0.3.0] - 2026-03-21
### Added
- **Go module proxy** — full GOPROXY protocol support (list, info, mod, zip, latest) (#59)
- **Upstream proxy retry** with configurable timeout and backoff (#56)
- **Maven proxy-only mode** — proxy Maven artifacts without local storage (#56)
- **Anonymous read mode** docs — Go proxy section in README (#62)
- Integration tests: Docker push/pull, npm install, upstream timeout (#57)
- Go proxy and Raw registry integration tests in smoke suite (#72)
- Config validation at startup — clear errors instead of runtime panics
- Dockerfile HEALTHCHECK for standalone deployments (#72)
- rust-toolchain.toml for reproducible builds (#72)
### Changed
- **Token hashing migrated from SHA-256 to Argon2id** — existing tokens auto-migrate on first use (#55)
- UI: Raw registry in sidebar, footer stats updated (32MB, 7 registries) (#64)
- README restructured: roadmap in README, removed stale ROADMAP.md (#65, #66)
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Remove all unwrap() from production code — proper error handling throughout (#72)
- Add `#![forbid(unsafe_code)]` — no unsafe code allowed at crate level (#72)
- Add input validation to Cargo registry endpoints (#72)
- Improve expect() messages with descriptive context (#72)
- Remove 7 unnecessary clone() calls (#72)
- Restore .gitleaks.toml lost during merge (#58)
- Update SECURITY.md — add 0.3.x to supported versions (#72)
### Security
- Update rustls-webpki 0.103.9 → 0.103.10 (RUSTSEC-2026-0049)
- Argon2id token hashing replaces SHA-256 (#55)
- `#![forbid(unsafe_code)]` enforced (#72)
- Zero unwrap() in production code (#72)
## [0.2.35] - 2026-03-20
### Added
- **Anonymous read mode** (`NORA_AUTH_ANONYMOUS_READ=true`): allow pull/download without credentials while requiring auth for push. Use case: public demo registries, read-only mirrors.
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Pin slsa-github-generator and codeql-action by SHA instead of tag
- Replace anonymous tuple with named struct in activity grouping (readability)
- Replace unwrap() with if-let pattern in activity grouping (safety)
- Add warning message on SLSA attestation failure instead of silent suppression
## [0.2.34] - 2026-03-20
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- **UI**: Group consecutive identical activity entries — repeated cache hits show as "artifact (x4)" instead of 4 identical rows
- **UI**: Fix table cell padding in Mount Points and Activity tables — th/td alignment now consistent
- **Security**: Update tar crate 0.4.44 → 0.4.45 (CVE-2026-33055 PAX size header bypass, CVE-2026-33056 symlink chmod traversal)
### Added
- 82 new unit tests across 7 modules (activity_log, audit, config, dashboard_metrics, error, metrics, repo_index)
- Test coverage badge in README (12.55% → 21.56%)
- Dashboard GIF (EN/RU crossfade) in README
- 7 missing environment variables added to docs (NORA_PUBLIC_URL, S3 credentials, NPM_METADATA_TTL, Raw config)
### Changed
- README restructured: tagline + docker run + GIF first, badges moved to Security section
- Remove hardcoded OpenSSF Scorecard version from README
## [0.2.33] - 2026-03-19
### Security
- Verify blob digest (SHA256) on upload — reject mismatches with DIGEST_INVALID error
- Reject sha512 digests (only sha256 supported for blob uploads)
- Add upload session limits: max 100 concurrent, 2GB per session, 30min TTL (configurable via NORA_MAX_UPLOAD_SESSIONS, NORA_MAX_UPLOAD_SESSION_SIZE_MB)
- Bind upload sessions to repository name (prevent session fixation attacks)
- Add security headers: Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy
- Run containers as non-root user (USER nora) in all Dockerfiles
### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Filter .meta.json from Docker tag list (fixes ArgoCD Image Updater tag recursion)
- Fix catalog endpoint to show namespaced images correctly (library/alpine instead of library)
### Added
- CodeQL workflow for SAST analysis
- SLSA provenance attestation for release artifacts
### Changed
- Configurable upload session size for ML models via NORA_MAX_UPLOAD_SESSION_SIZE_MB (default 2048 MB)
## [0.2.32] - 2026-03-18
### Fixed / Исправлено
- **Docker dashboard**: Namespaced images (library/alpine, grafana/grafana) now visible in UI — index builder finds manifests by position, not fixed index
- **Docker proxy**: Auto-prepend `library/` for single-segment official Hub images (nginx, alpine, node) — no need to explicitly use library/ prefix
- **CI**: Fixed cargo-deny license checks (NCSA for libfuzzer-sys, MIT for fuzz crate, unused-allowed-license config)
- **Docker dashboard**: Namespaced-образы (library/alpine, grafana/grafana) теперь отображаются в UI
- **Docker proxy**: Автоподстановка `library/` для официальных образов Docker Hub (nginx, alpine, node) — больше не нужно указывать library/ вручную
- **CI**: Исправлены проверки лицензий cargo-deny
## [0.2.31] - 2026-03-16
### Added / Добавлено
- **npm URL rewriting**: Tarball URLs in proxied metadata now rewritten to point to NORA (previously tarballs bypassed NORA and downloaded directly from npmjs.org)
- **npm scoped packages**: Full support for `@scope/package` in proxy handler and repository index
- **npm publish**: `PUT /npm/{package}` accepts standard npm publish payload with base64-encoded tarballs
- **npm metadata TTL**: Configurable cache TTL (`NORA_NPM_METADATA_TTL`, default 300s) with stale-while-revalidate fallback
- **Immutable cache**: SHA256 integrity verification on cached npm tarballs — detects tampering on cache hit
- **npm URL rewriting**: Tarball URL в проксированных метаданных теперь переписываются на NORA (ранее тарболы шли напрямую из npmjs.org)
- **npm scoped packages**: Полная поддержка `@scope/package` в прокси-хендлере и индексе репозитория
- **npm publish**: `PUT /npm/{package}` принимает стандартный npm publish payload с base64-тарболами
- **npm metadata TTL**: Настраиваемый TTL кеша (`NORA_NPM_METADATA_TTL`, default 300s) с stale-while-revalidate
- **Immutable cache**: SHA256 проверка целостности npm-тарболов — обнаружение подмены при отдаче из кеша
### Security / Безопасность
- **Path traversal protection**: Attachment filename validation in npm publish (rejects `../`, `/`, `\`)
- **Package name mismatch**: npm publish rejects payloads where URL path doesn't match `name` field (anti-spoofing)
- **Version immutability**: npm publish returns 409 Conflict on duplicate version
- **Защита от path traversal**: Валидация имён файлов в npm publish (отклоняет `../`, `/`, `\`)
- **Проверка имени пакета**: npm publish отклоняет payload если имя в URL не совпадает с полем `name` (anti-spoofing)
- **Иммутабельность версий**: npm publish возвращает 409 Conflict при попытке перезаписать версию
### Fixed / Исправлено
- **npm proxy_auth**: `proxy_auth` field was configured but not wired into `fetch_from_proxy` — now sends Basic Auth header to upstream
- **npm proxy_auth**: Поле `proxy_auth` было в конфиге, но не передавалось в `fetch_from_proxy` — теперь отправляет Basic Auth в upstream
All notable changes to NORA will be documented in this file. All notable changes to NORA will be documented in this file.
@@ -609,10 +386,6 @@ All notable changes to NORA will be documented in this file.
## [0.2.15] - 2026-01-31 ## [0.2.15] - 2026-01-31
### Fixed ### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Code formatting (cargo fmt) - Code formatting (cargo fmt)
--- ---
@@ -630,10 +403,6 @@ All notable changes to NORA will be documented in this file.
## [0.2.14] - 2026-01-31 ## [0.2.14] - 2026-01-31
### Fixed ### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Docker dashboard now shows actual image size from manifest layers (config + layers sum) - Docker dashboard now shows actual image size from manifest layers (config + layers sum)
- Previously showed only manifest file size (~500 B instead of actual image size) - Previously showed only manifest file size (~500 B instead of actual image size)
@@ -652,10 +421,6 @@ All notable changes to NORA will be documented in this file.
## [0.2.13] - 2026-01-31 ## [0.2.13] - 2026-01-31
### Fixed ### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- npm dashboard now shows correct version count and package sizes - npm dashboard now shows correct version count and package sizes
- Parses metadata.json for versions, dist.unpackedSize, and time.modified - Parses metadata.json for versions, dist.unpackedSize, and time.modified
- Previously showed 0 versions / 0 B for all packages - Previously showed 0 versions / 0 B for all packages
@@ -820,10 +585,6 @@ All notable changes to NORA will be documented in this file.
## [0.2.5] - 2026-01-26 ## [0.2.5] - 2026-01-26
### Fixed ### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Docker push/pull: added PATCH endpoint for chunked uploads - Docker push/pull: added PATCH endpoint for chunked uploads
--- ---
@@ -841,10 +602,6 @@ All notable changes to NORA will be documented in this file.
## [0.2.4] - 2026-01-26 ## [0.2.4] - 2026-01-26
### Fixed ### Fixed
- Go and Raw registries missing from Prometheus metrics (`detect_registry` labeled both as "other") (PR #97, @TickTockBent)
- Go and Raw registries missing from `/health` endpoint `registries` object (PR #97, @TickTockBent)
- Garbage collection scoped to Docker-only blobs — prevents GC from deleting non-Docker registry data (PR #109, @TickTockBent)
- Correct `zeroize` annotation placement and avoid secret cloning in `protected.rs` (PR #108, @TickTockBent)
- Rate limiting: health/metrics endpoints now exempt - Rate limiting: health/metrics endpoints now exempt
- Increased upload rate limits for Docker parallel requests - Increased upload rate limits for Docker parallel requests

View File

@@ -1,36 +0,0 @@
# Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity and
orientation.
## Our Standards
Examples of behavior that contributes to a positive environment:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
Examples of unacceptable behavior:
* The use of sexualized language or imagery, and sexual attention or advances
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information without explicit permission
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the project team at security@getnora.io. All complaints will be
reviewed and investigated promptly and fairly.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/), version 2.1.

149
COMPAT.md
View File

@@ -1,149 +0,0 @@
# NORA Registry Protocol Compatibility
This document describes which parts of each registry protocol are implemented in NORA.
**Legend:** Full = complete implementation, Partial = basic support with limitations, Stub = placeholder, — = not implemented
## Docker (OCI Distribution Spec 1.1)
| Endpoint | Method | Status | Notes |
|----------|--------|--------|-------|
| `/v2/` | GET | Full | API version check |
| `/v2/_catalog` | GET | Full | List all repositories |
| `/v2/{name}/tags/list` | GET | Full | List image tags |
| `/v2/{name}/manifests/{ref}` | GET | Full | By tag or digest |
| `/v2/{name}/manifests/{ref}` | HEAD | Full | Check manifest exists |
| `/v2/{name}/manifests/{ref}` | PUT | Full | Push manifest |
| `/v2/{name}/manifests/{ref}` | DELETE | Full | Delete manifest |
| `/v2/{name}/blobs/{digest}` | GET | Full | Download layer/config |
| `/v2/{name}/blobs/{digest}` | HEAD | Full | Check blob exists |
| `/v2/{name}/blobs/{digest}` | DELETE | Full | Delete blob |
| `/v2/{name}/blobs/uploads/` | POST | Full | Start chunked upload |
| `/v2/{name}/blobs/uploads/{uuid}` | PATCH | Full | Upload chunk |
| `/v2/{name}/blobs/uploads/{uuid}` | PUT | Full | Complete upload |
| Namespaced `{ns}/{name}` | * | Full | Two-level paths |
| Deep paths `a/b/c/name` | * | — | Max 2-level (`org/image`) |
| Token auth (Bearer) | — | Full | WWW-Authenticate challenge |
| Cross-repo blob mount | POST | — | Not implemented |
| Referrers API | GET | — | OCI 1.1 referrers |
### Known Limitations
- Max 2-level image path: `org/image:tag` works, `org/sub/path/image:tag` returns 404
- Large monolithic blob PUT (>~500MB) may fail even with high body limit
- No cross-repository blob mounting
## npm
| Feature | Status | Notes |
|---------|--------|-------|
| Package metadata (GET) | Full | JSON with all versions |
| Scoped packages `@scope/name` | Full | URL-encoded path |
| Tarball download | Full | SHA256 verified |
| Tarball URL rewriting | Full | Points to NORA, not upstream |
| Publish (`npm publish`) | Full | Immutable versions |
| Unpublish | — | Not implemented |
| Dist-tags (`latest`, `next`) | Partial | Read from metadata, no explicit management |
| Search (`/-/v1/search`) | — | Not implemented |
| Audit (`/-/npm/v1/security/advisories`) | — | Not implemented |
| Upstream proxy | Full | Configurable TTL |
## Maven
| Feature | Status | Notes |
|---------|--------|-------|
| Artifact download (GET) | Full | JAR, POM, checksums |
| Artifact upload (PUT) | Full | Any file type |
| GroupId path layout | Full | Dots → slashes |
| SHA1/MD5 checksums | Full | Stored alongside artifacts |
| `maven-metadata.xml` | Partial | Stored as-is, no auto-generation |
| SNAPSHOT versions | — | No SNAPSHOT resolution |
| Multi-proxy fallback | Full | Tries proxies in order |
| Content-Type by extension | Full | .jar, .pom, .xml, .sha1, .md5 |
### Known Limitations
- `maven-metadata.xml` not auto-generated on publish (must be uploaded explicitly)
- No SNAPSHOT version management (`-SNAPSHOT` → latest timestamp)
## Cargo (Sparse Index, RFC 2789)
| Feature | Status | Notes |
|---------|--------|-------|
| `config.json` | Full | `dl` and `api` fields |
| Sparse index lookup | Full | Prefix rules (1/2/3/ab/cd) |
| Crate download | Full | `.crate` files by version |
| `cargo publish` | Full | Length-prefixed JSON + .crate |
| Dependency metadata | Full | `req`, `package` transforms |
| SHA256 verification | Full | On publish |
| Cache-Control headers | Full | `immutable` for downloads, `max-age=300` for index |
| Yank/unyank | — | Not implemented |
| Owner management | — | Not implemented |
| Categories/keywords | Partial | Stored but not searchable |
## PyPI (PEP 503/691)
| Feature | Status | Notes |
|---------|--------|-------|
| Simple index (HTML) | Full | PEP 503 |
| Simple index (JSON) | Full | PEP 691, via Accept header |
| Package versions page | Full | HTML + JSON |
| File download | Full | Wheel, sdist, egg |
| `twine upload` | Full | Multipart form-data |
| SHA256 hashes | Full | In metadata links |
| Case normalization | Full | `My-Package``my-package` |
| Upstream proxy | Full | Configurable TTL |
| JSON API metadata | Full | `application/vnd.pypi.simple.v1+json` |
| Yanking | — | Not implemented |
| Upload signatures (PGP) | — | Not implemented |
## Go Module Proxy (GOPROXY)
| Feature | Status | Notes |
|---------|--------|-------|
| `/@v/list` | Full | List known versions |
| `/@v/{version}.info` | Full | Version metadata JSON |
| `/@v/{version}.mod` | Full | go.mod file |
| `/@v/{version}.zip` | Full | Module zip archive |
| `/@latest` | Full | Latest version info |
| Module path escaping | Full | `!x``X` per spec |
| Immutability | Full | .info, .mod, .zip immutable after first write |
| Size limit for .zip | Full | Configurable |
| `$GONOSUMDB` / `$GONOSUMCHECK` | — | Not relevant (client-side) |
| Upstream proxy | — | Direct storage only |
## Raw File Storage
| Feature | Status | Notes |
|---------|--------|-------|
| Upload (PUT) | Full | Any file type |
| Download (GET) | Full | Content-Type by extension |
| Delete (DELETE) | Full | |
| Exists check (HEAD) | Full | Returns size + Content-Type |
| Max file size | Full | Configurable (default 1MB) |
| Directory listing | — | Not implemented |
| Versioning | — | Overwrite-only |
## Helm OCI
Helm charts are stored as OCI artifacts via the Docker registry endpoints. `helm push` and `helm pull` work through the standard `/v2/` API.
| Feature | Status | Notes |
|---------|--------|-------|
| `helm push` (OCI) | Full | Via Docker PUT manifest/blob |
| `helm pull` (OCI) | Full | Via Docker GET manifest/blob |
| Helm repo index (`index.yaml`) | — | Not implemented (OCI only) |
## Cross-Cutting Features
| Feature | Status | Notes |
|---------|--------|-------|
| Authentication (Bearer/Basic) | Full | Per-request token validation |
| Anonymous read | Full | `NORA_AUTH_ANONYMOUS_READ=true` |
| Rate limiting | Full | `tower_governor`, per-IP |
| Prometheus metrics | Full | `/metrics` endpoint |
| Health check | Full | `/health` |
| Swagger/OpenAPI | Full | `/swagger-ui/` |
| S3 backend | Full | AWS, MinIO, any S3-compatible |
| Local filesystem backend | Full | Default, content-addressable |
| Activity log | Full | Recent push/pull in dashboard |
| Backup/restore | Full | CLI commands |
| Mirror CLI | Full | `nora mirror` for npm/pip/cargo/maven/docker |

View File

@@ -2,34 +2,6 @@
Thank you for your interest in contributing to NORA! Thank you for your interest in contributing to NORA!
## Developer Certificate of Origin (DCO)
By submitting a pull request, you agree to the [Developer Certificate of Origin](https://developercertificate.org/).
Your contribution will be licensed under the [MIT License](LICENSE).
You confirm that you have the right to submit the code and that it does not violate any third-party rights.
## Project Governance
NORA uses a **Benevolent Dictator** governance model:
- **Maintainer:** [@devitway](https://github.com/devitway) — final decisions on features, releases, and architecture
- **Contributors:** anyone who submits issues, PRs, or docs improvements
- **Decision process:** proposals via GitHub Issues → discussion → maintainer decision
- **Release authority:** maintainer only
### Roles and Responsibilities
| Role | Person | Responsibilities |
|------|--------|-----------------|
| Maintainer | @devitway | Code review, releases, roadmap, security response |
| Contributor | anyone | Issues, PRs, documentation, testing |
| Dependabot | automated | Dependency updates |
### Continuity
The GitHub organization [getnora-io](https://github.com/getnora-io) has multiple admin accounts to ensure project continuity. Source code is MIT-licensed, enabling anyone to fork and continue the project.
## Getting Started ## Getting Started
1. Fork the repository 1. Fork the repository
@@ -38,110 +10,62 @@ The GitHub organization [getnora-io](https://github.com/getnora-io) has multiple
## Development Setup ## Development Setup
### Prerequisites
- **Rust** stable (1.85+) — install via [rustup](https://rustup.rs/)
- **Docker** (optional) — for integration tests (docker push/pull)
- **Node.js** 18+ (optional) — for npm integration tests
### Build and Test
```bash ```bash
# Install Rust (if needed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Enable pre-commit hooks (important!)
git config core.hooksPath .githooks
# Build # Build
cargo build --package nora-registry cargo build
# Run unit tests (important: use --lib --bin to skip fuzz targets) # Run tests
cargo test --lib --bin nora cargo test
# Run clippy (must pass with zero warnings) # Run locally
cargo clippy --package nora-registry -- -D warnings
# Format check
cargo fmt --check
```
### Run Locally
```bash
# Start with defaults (port 4000, local storage in ./data/)
cargo run --bin nora -- serve cargo run --bin nora -- serve
# Custom port and storage
NORA_PORT=5000 NORA_STORAGE_PATH=/tmp/nora-data cargo run --bin nora -- serve
# Test health
curl http://localhost:4000/health
``` ```
### Integration / Smoke Tests
```bash
# Build release binary first
cargo build --release
# Run full smoke suite (starts NORA, tests all 7 protocols, stops)
bash tests/smoke.sh
```
### Fuzz Testing
```bash
# Install cargo-fuzz (one-time)
cargo install cargo-fuzz
# Run fuzz target (Ctrl+C to stop)
cargo +nightly fuzz run fuzz_validation -- -max_total_time=60
```
## Before Submitting a PR
```bash
cargo fmt --check
cargo clippy --package nora-registry -- -D warnings
cargo test --lib --bin nora
```
All three must pass. CI will enforce this.
## Code Style ## Code Style
- Run `cargo fmt` before committing - Run `cargo fmt` before committing
- Fix all `cargo clippy` warnings - Run `cargo clippy` and fix warnings
- Follow Rust naming conventions - Follow Rust naming conventions
- Keep functions short and focused
- Add tests for new functionality
## Pull Request Process ## Pull Request Process
1. Update CHANGELOG.md if the change is user-facing 1. Update documentation if needed
2. Add tests for new features or bug fixes 2. Add tests for new features
3. Ensure CI passes (fmt, clippy, test, security checks) 3. Ensure all tests pass: `cargo test`
4. Keep PRs focused — one feature or fix per PR 4. Ensure code is formatted: `cargo fmt --check`
5. Ensure no clippy warnings: `cargo clippy`
## Commit Messages ## Commit Messages
Use conventional commits: Use conventional commits:
- `feat:` new feature - `feat:` - new feature
- `fix:` bug fix - `fix:` - bug fix
- `docs:` documentation - `docs:` - documentation
- `test:` adding or updating tests - `style:` - formatting
- `security:` security improvements - `refactor:` - code refactoring
- `chore:` maintenance - `test:` - adding tests
- `chore:` - maintenance
Example: `feat: add npm scoped package support` Example: `feat: add OAuth2 authentication`
## Reporting Issues ## Reporting Issues
- Use GitHub Issues with the provided templates - Use GitHub Issues
- Include steps to reproduce - Include steps to reproduce
- Include NORA version (`nora --version`) and OS - Include NORA version and OS
## License ## License
By contributing, you agree that your contributions will be licensed under the MIT License. By contributing, you agree that your contributions will be licensed under the MIT License.
## Community ## Contact
- Telegram: [@getnora](https://t.me/getnora) - Telegram: [@DevITWay](https://t.me/DevITWay)
- GitHub Issues: [getnora-io/nora](https://github.com/getnora-io/nora/issues) - GitHub Issues: [getnora-io/nora](https://github.com/getnora-io/nora/issues)

616
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -2,13 +2,13 @@
resolver = "2" resolver = "2"
members = [ members = [
"nora-registry", "nora-registry",
"fuzz", "nora-storage",
"nora-cli",
] ]
[workspace.package] [workspace.package]
version = "0.5.0" version = "0.2.31"
edition = "2021" edition = "2021"
rust-version = "1.75"
license = "MIT" license = "MIT"
authors = ["DevITWay <devitway@gmail.com>"] authors = ["DevITWay <devitway@gmail.com>"]
repository = "https://github.com/getnora-io/nora" repository = "https://github.com/getnora-io/nora"
@@ -16,37 +16,13 @@ homepage = "https://getnora.io"
[workspace.dependencies] [workspace.dependencies]
tokio = { version = "1", features = ["full"] } tokio = { version = "1", features = ["full"] }
axum = { version = "0.8", features = ["multipart"] } axum = "0.8"
serde = { version = "1", features = ["derive"] } serde = { version = "1", features = ["derive"] }
serde_json = "1" serde_json = "1"
tracing = "0.1" tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] } tracing-subscriber = { version = "0.3", features = ["env-filter", "json"] }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] } reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
sha2 = "0.11" sha2 = "0.10"
async-trait = "0.1" async-trait = "0.1"
hmac = "0.13" hmac = "0.12"
hex = "0.4" hex = "0.4"
[workspace.lints.clippy]
or_fun_call = "deny"
redundant_clone = "deny"
collection_is_never_read = "deny"
naive_bytecount = "deny"
stable_sort_primitive = "deny"
large_types_passed_by_value = "deny"
assigning_clones = "deny"
[workspace.lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = [] }
[profile.release]
debug = "line-tables-only"
codegen-units = 4
panic = "abort"
lto = "thin"
# Maximum optimization for GitHub Releases and published binaries
[profile.release-official]
inherits = "release"
codegen-units = 1
lto = true

View File

@@ -1,12 +1,10 @@
# syntax=docker/dockerfile:1.4 # syntax=docker/dockerfile:1.4
# Binary is pre-built by CI (cargo build --release) and passed via context # Binary is pre-built by CI (cargo build --release) and passed via context
FROM alpine:3.20@sha256:a4f4213abb84c497377b8544c81b3564f313746700372ec4fe84653e4fb03805 FROM alpine:3.20
RUN apk add --no-cache ca-certificates \ RUN apk add --no-cache ca-certificates && mkdir -p /data
&& addgroup -S nora && adduser -S -G nora nora \
&& mkdir -p /data && chown nora:nora /data
COPY --chown=nora:nora nora /usr/local/bin/nora COPY nora /usr/local/bin/nora
ENV RUST_LOG=info ENV RUST_LOG=info
ENV NORA_HOST=0.0.0.0 ENV NORA_HOST=0.0.0.0
@@ -19,10 +17,5 @@ EXPOSE 4000
VOLUME ["/data"] VOLUME ["/data"]
USER nora
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD wget -q --spider http://localhost:4000/health || exit 1
ENTRYPOINT ["/usr/local/bin/nora"] ENTRYPOINT ["/usr/local/bin/nora"]
CMD ["serve"] CMD ["serve"]

View File

@@ -1,15 +1,17 @@
# syntax=docker/dockerfile:1.4 # syntax=docker/dockerfile:1.4
# NORA on Astra Linux SE base (Debian-based, FSTEC-certified) # Binary is pre-built by CI (cargo build --release) and passed via context
# Binary is pre-built by CI and passed via context # Runtime: scratch — compatible with Astra Linux SE (FSTEC certified)
FROM debian:bookworm-slim@sha256:f06537653ac770703bc45b4b113475bd402f451e85223f0f2837acbf89ab020a # To switch to official base: replace FROM scratch with
# FROM registry.astralinux.ru/library/alse:latest
# RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/*
RUN apt-get update \ FROM alpine:3.20 AS certs
&& apt-get install -y --no-install-recommends ca-certificates curl \ RUN apk add --no-cache ca-certificates
&& rm -rf /var/lib/apt/lists/* \
&& groupadd -r nora && useradd -r -g nora -d /data -s /usr/sbin/nologin nora \
&& mkdir -p /data && chown nora:nora /data
COPY --chown=nora:nora nora /usr/local/bin/nora FROM scratch
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY nora /usr/local/bin/nora
ENV RUST_LOG=info ENV RUST_LOG=info
ENV NORA_HOST=0.0.0.0 ENV NORA_HOST=0.0.0.0
@@ -22,10 +24,5 @@ EXPOSE 4000
VOLUME ["/data"] VOLUME ["/data"]
USER nora
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD curl -sf http://localhost:4000/health || exit 1
ENTRYPOINT ["/usr/local/bin/nora"] ENTRYPOINT ["/usr/local/bin/nora"]
CMD ["serve"] CMD ["serve"]

View File

@@ -1,14 +1,17 @@
# syntax=docker/dockerfile:1.4 # syntax=docker/dockerfile:1.4
# NORA on RED OS base (RPM-based, FSTEC-certified) # Binary is pre-built by CI (cargo build --release) and passed via context
# Binary is pre-built by CI and passed via context # Runtime: scratch — compatible with RED OS (FSTEC certified)
FROM registry.access.redhat.com/ubi9/ubi-minimal:9.4@sha256:c0e70387664f30cd9cf2795b547e4a9a51002c44a4a86aa9335ab030134bf392 # To switch to official base: replace FROM scratch with
# FROM registry.red-soft.ru/redos/redos:8
# RUN dnf install -y ca-certificates && dnf clean all
RUN microdnf install -y ca-certificates shadow-utils \ FROM alpine:3.20 AS certs
&& microdnf clean all \ RUN apk add --no-cache ca-certificates
&& groupadd -r nora && useradd -r -g nora -d /data -s /sbin/nologin nora \
&& mkdir -p /data && chown nora:nora /data
COPY --chown=nora:nora nora /usr/local/bin/nora FROM scratch
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY nora /usr/local/bin/nora
ENV RUST_LOG=info ENV RUST_LOG=info
ENV NORA_HOST=0.0.0.0 ENV NORA_HOST=0.0.0.0
@@ -21,10 +24,5 @@ EXPOSE 4000
VOLUME ["/data"] VOLUME ["/data"]
USER nora
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD curl -sf http://localhost:4000/health || exit 1
ENTRYPOINT ["/usr/local/bin/nora"] ENTRYPOINT ["/usr/local/bin/nora"]
CMD ["serve"] CMD ["serve"]

203
README.md
View File

@@ -1,44 +1,45 @@
# NORA
**The artifact registry that grows with you.** Starts with `docker run`, scales to enterprise.
```bash
docker run -d -p 4000:4000 -v nora-data:/data ghcr.io/getnora-io/nora:latest
```
Open [http://localhost:4000/ui/](http://localhost:4000/ui/) — your registry is ready.
<p align="center">
<img src=".github/assets/dashboard.png" alt="NORA Dashboard" width="960" />
</p>
## Why NORA
- **Zero-config** — single 32 MB binary, no database, no dependencies. `docker run` and it works.
- **Production-tested** — Docker (+ Helm OCI), Maven, npm, PyPI, Cargo, Go, Raw. Used in real CI/CD with ArgoCD, Buildx cache, and air-gapped environments.
- **Secure by default** — [OpenSSF Scorecard](https://scorecard.dev/viewer/?uri=github.com/getnora-io/nora), signed releases, SBOM, fuzz testing, 460+ tests.
[![Release](https://img.shields.io/github/v/release/getnora-io/nora)](https://github.com/getnora-io/nora/releases)
[![Image Size](https://img.shields.io/badge/image-32%20MB-blue)](https://github.com/getnora-io/nora/pkgs/container/nora)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE) [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Release](https://img.shields.io/github/v/release/getnora-io/nora)](https://github.com/getnora-io/nora/releases)
[![CI](https://img.shields.io/github/actions/workflow/status/getnora-io/nora/ci.yml?label=CI)](https://github.com/getnora-io/nora/actions)
[![GHCR](https://img.shields.io/badge/ghcr.io-nora-blue?logo=github)](https://github.com/getnora-io/nora/pkgs/container/nora)
[![GitHub Stars](https://img.shields.io/github/stars/getnora-io/nora?style=flat&logo=github)](https://github.com/getnora-io/nora/stargazers)
[![Rust](https://img.shields.io/badge/rust-%23000000.svg?logo=rust&logoColor=white)](https://www.rust-lang.org/)
[![Docs](https://img.shields.io/badge/docs-getnora.dev-green?logo=gitbook)](https://getnora.dev)
[![Telegram](https://img.shields.io/badge/Telegram-Community-blue?logo=telegram)](https://t.me/getnora)
**32 MB** binary | **< 100 MB** RAM | **3s** startup | **7** registries > **Multi-protocol artifact registry that doesn't suck.**
>
> One binary. All protocols. Stupidly fast.
> Used in production at [DevIT Academy](https://github.com/devitway) since January 2026 for Docker images, Maven artifacts, and npm packages. **32 MB** binary | **< 100 MB** RAM | **3s** startup | **5** protocols
## Supported Registries ## Features
| Registry | Mount Point | Upstream Proxy | Auth | - **Multi-Protocol Support**
|----------|------------|----------------|------| - Docker Registry v2
| Docker Registry v2 | `/v2/` | Docker Hub, GHCR, any OCI, Helm OCI | ✓ | - Maven repository (+ proxy to Maven Central)
| Maven | `/maven2/` | Maven Central, custom | proxy-only | - npm registry (+ proxy to npmjs.org)
| npm | `/npm/` | npmjs.org, custom | ✓ | - Cargo registry
| Cargo | `/cargo/` | — | ✓ | - PyPI index
| PyPI | `/simple/` | pypi.org, custom | ✓ |
| Go Modules | `/go/` | proxy.golang.org, custom | ✓ |
| Raw files | `/raw/` | — | ✓ |
> **Helm charts** work via the Docker/OCI endpoint — `helm push`/`pull` with `--plain-http` or behind TLS reverse proxy. - **Storage Backends**
- Local filesystem (zero-config default)
- S3-compatible (MinIO, AWS S3)
- **Production Ready**
- Web UI with search and browse
- Swagger UI API documentation
- Prometheus metrics (`/metrics`)
- Health checks (`/health`, `/ready`)
- JSON structured logging
- Graceful shutdown
- **Security**
- Basic Auth (htpasswd + bcrypt)
- Revocable API tokens with RBAC
- ENV-based configuration (12-Factor)
- SBOM (SPDX + CycloneDX) in every release
- See [SECURITY.md](SECURITY.md) for vulnerability reporting
## Quick Start ## Quick Start
@@ -48,13 +49,6 @@ Open [http://localhost:4000/ui/](http://localhost:4000/ui/) — your registry is
docker run -d -p 4000:4000 -v nora-data:/data ghcr.io/getnora-io/nora:latest docker run -d -p 4000:4000 -v nora-data:/data ghcr.io/getnora-io/nora:latest
``` ```
### Binary
```bash
curl -fsSL https://github.com/getnora-io/nora/releases/latest/download/nora-linux-amd64 -o nora
chmod +x nora && ./nora
```
### From Source ### From Source
```bash ```bash
@@ -62,13 +56,18 @@ cargo install nora-registry
nora nora
``` ```
Open http://localhost:4000/ui/
## Usage ## Usage
### Docker Images ### Docker Images
```bash ```bash
# Tag and push
docker tag myapp:latest localhost:4000/myapp:latest docker tag myapp:latest localhost:4000/myapp:latest
docker push localhost:4000/myapp:latest docker push localhost:4000/myapp:latest
# Pull
docker pull localhost:4000/myapp:latest docker pull localhost:4000/myapp:latest
``` ```
@@ -89,42 +88,31 @@ npm config set registry http://localhost:4000/npm/
npm publish npm publish
``` ```
### Go Modules
```bash
GOPROXY=http://localhost:4000/go go get golang.org/x/text@latest
```
## Features
- **Web UI** — dashboard with search, browse, i18n (EN/RU)
- **Proxy & Cache** — transparent proxy to upstream registries with local cache
- **Mirror CLI** — offline sync for air-gapped environments (`nora mirror`)
- **Backup & Restore** — `nora backup` / `nora restore`
- **Migration** — `nora migrate --from local --to s3`
- **S3 Storage** — MinIO, AWS S3, any S3-compatible backend
- **Prometheus Metrics** — `/metrics` endpoint
- **Health Checks** — `/health`, `/ready` for Kubernetes probes
- **Swagger UI** — `/api-docs` for API exploration
- **Rate Limiting** — configurable per-endpoint rate limits
- **FSTEC Builds** — Astra Linux SE and RED OS images in every release
## Authentication ## Authentication
NORA supports Basic Auth (htpasswd) and revocable API tokens with RBAC. NORA supports Basic Auth (htpasswd) and revocable API tokens with RBAC.
```bash ### Quick Setup
# Create htpasswd file
htpasswd -cbB users.htpasswd admin yourpassword
# Start with auth enabled ```bash
# 1. Create htpasswd file with bcrypt
htpasswd -cbB users.htpasswd admin yourpassword
# Add more users:
htpasswd -bB users.htpasswd ci-user ci-secret
# 2. Start NORA with auth enabled
docker run -d -p 4000:4000 \ docker run -d -p 4000:4000 \
-v nora-data:/data \ -v nora-data:/data \
-v ./users.htpasswd:/data/users.htpasswd \ -v ./users.htpasswd:/data/users.htpasswd \
-e NORA_AUTH_ENABLED=true \ -e NORA_AUTH_ENABLED=true \
ghcr.io/getnora-io/nora:latest ghcr.io/getnora-io/nora:latest
# 3. Verify
curl -u admin:yourpassword http://localhost:4000/v2/_catalog
``` ```
### API Tokens (RBAC)
| Role | Pull/Read | Push/Write | Delete/Admin | | Role | Pull/Read | Push/Write | Delete/Admin |
|------|-----------|------------|--------------| |------|-----------|------------|--------------|
| `read` | Yes | No | No | | `read` | Yes | No | No |
@@ -133,6 +121,16 @@ docker run -d -p 4000:4000 \
See [Authentication guide](https://getnora.dev/configuration/authentication/) for token management, Docker login, and CI/CD integration. See [Authentication guide](https://getnora.dev/configuration/authentication/) for token management, Docker login, and CI/CD integration.
## CLI Commands
```bash
nora # Start server
nora serve # Start server (explicit)
nora backup -o backup.tar.gz
nora restore -i backup.tar.gz
nora migrate --from local --to s3
```
## Configuration ## Configuration
### Environment Variables ### Environment Variables
@@ -144,10 +142,8 @@ See [Authentication guide](https://getnora.dev/configuration/authentication/) fo
| `NORA_STORAGE_MODE` | local | `local` or `s3` | | `NORA_STORAGE_MODE` | local | `local` or `s3` |
| `NORA_AUTH_ENABLED` | false | Enable authentication | | `NORA_AUTH_ENABLED` | false | Enable authentication |
| `NORA_DOCKER_UPSTREAMS` | `https://registry-1.docker.io` | Docker upstreams (`url\|user:pass,...`) | | `NORA_DOCKER_UPSTREAMS` | `https://registry-1.docker.io` | Docker upstreams (`url\|user:pass,...`) |
| `NORA_LOG_LEVEL` | info | Log level: trace, debug, info, warn, error |
| `NORA_LOG_FORMAT` | text | Log format: `text` (human) or `json` (structured) | See [full configuration reference](https://getnora.dev/configuration/settings/) for all environment variables including storage, rate limiting, proxy auth, and secrets.
| `NORA_PUBLIC_URL` | — | Public URL for rewriting artifact links |
See [full configuration reference](https://getnora.dev/configuration/settings/) for all options.
### config.toml ### config.toml
@@ -169,21 +165,9 @@ proxy_timeout = 60
[[docker.upstreams]] [[docker.upstreams]]
url = "https://registry-1.docker.io" url = "https://registry-1.docker.io"
[go]
proxy = "https://proxy.golang.org"
``` ```
## CLI Commands See [full config reference](https://getnora.dev/configuration/settings/) for rate limiting, secrets, proxy auth, and all options.
```bash
nora # Start server
nora serve # Start server (explicit)
nora backup -o backup.tar.gz
nora restore -i backup.tar.gz
nora migrate --from local --to s3
nora mirror # Sync packages for offline use
```
## Endpoints ## Endpoints
@@ -199,7 +183,6 @@ nora mirror # Sync packages for offline use
| `/npm/` | npm | | `/npm/` | npm |
| `/cargo/` | Cargo | | `/cargo/` | Cargo |
| `/simple/` | PyPI | | `/simple/` | PyPI |
| `/go/` | Go Modules |
## TLS / HTTPS ## TLS / HTTPS
@@ -211,8 +194,21 @@ registry.example.com {
} }
``` ```
For internal networks without TLS, configure Docker:
```json
// /etc/docker/daemon.json
{
"insecure-registries": ["192.168.1.100:4000"]
}
```
See [TLS / HTTPS guide](https://getnora.dev/configuration/tls/) for Nginx, Traefik, and custom CA setup. See [TLS / HTTPS guide](https://getnora.dev/configuration/tls/) for Nginx, Traefik, and custom CA setup.
## FSTEC-Certified OS Builds
Dedicated builds for Astra Linux SE and RED OS are published as `-astra` and `-redos` tagged images in every [GitHub Release](https://github.com/getnora-io/nora/releases). Both use `scratch` base with statically-linked binary.
## Performance ## Performance
| Metric | NORA | Nexus | JFrog | | Metric | NORA | Nexus | JFrog |
@@ -221,47 +217,24 @@ See [TLS / HTTPS guide](https://getnora.dev/configuration/tls/) for Nginx, Traef
| Memory | < 100 MB | 2-4 GB | 2-4 GB | | Memory | < 100 MB | 2-4 GB | 2-4 GB |
| Image Size | 32 MB | 600+ MB | 1+ GB | | Image Size | 32 MB | 600+ MB | 1+ GB |
[See how NORA compares to other registries](https://getnora.dev)
## Roadmap ## Roadmap
- **Mirror CLI** — offline sync for air-gapped environments
- **OIDC / Workload Identity** — zero-secret auth for GitHub Actions, GitLab CI - **OIDC / Workload Identity** — zero-secret auth for GitHub Actions, GitLab CI
- **Online Garbage Collection** — non-blocking cleanup without registry downtime - **Online Garbage Collection** — non-blocking cleanup without registry downtime
- **Retention Policies** — declarative rules: keep last N tags, delete older than X days - **Retention Policies** — declarative rules: keep last N tags, delete older than X days
- **Image Signing** — cosign verification and policy enforcement - **Image Signing** — cosign/notation verification and policy enforcement
- **Replication** — push/pull sync between NORA instances
See [CHANGELOG.md](CHANGELOG.md) for release history. See [CHANGELOG.md](CHANGELOG.md) for release history.
## Security & Trust
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/getnora-io/nora/badge)](https://scorecard.dev/viewer/?uri=github.com/getnora-io/nora)
[![CII Best Practices](https://www.bestpractices.dev/projects/12207/badge)](https://www.bestpractices.dev/projects/12207)
[![Coverage](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/devitway/0f0538f1ed16d5d9951e4f2d3f79b699/raw/nora-coverage.json)](https://github.com/getnora-io/nora/actions/workflows/ci.yml)
[![CI](https://img.shields.io/github/actions/workflow/status/getnora-io/nora/ci.yml?label=CI)](https://github.com/getnora-io/nora/actions)
- **Signed releases** — every release is signed with [cosign](https://github.com/sigstore/cosign)
- **SBOM** — SPDX + CycloneDX in every release
- **Fuzz testing** — cargo-fuzz + ClusterFuzzLite
- **Blob verification** — SHA256 digest validation on every upload
- **Non-root containers** — all images run as non-root
- **Security headers** — CSP, X-Frame-Options, nosniff
See [SECURITY.md](SECURITY.md) for vulnerability reporting.
## Author ## Author
Created and maintained by [DevITWay](https://github.com/devitway) **Created and maintained by [DevITWay](https://github.com/devitway)**
[![GHCR](https://img.shields.io/badge/ghcr.io-nora-blue?logo=github)](https://github.com/getnora-io/nora/pkgs/container/nora)
[![Rust](https://img.shields.io/badge/rust-%23000000.svg?logo=rust&logoColor=white)](https://www.rust-lang.org/)
[![Docs](https://img.shields.io/badge/docs-getnora.dev-green?logo=gitbook)](https://getnora.dev)
[![Telegram](https://img.shields.io/badge/Telegram-Community-blue?logo=telegram)](https://t.me/getnora)
[![GitHub Stars](https://img.shields.io/github/stars/getnora-io/nora?style=flat&logo=github)](https://github.com/getnora-io/nora/stargazers)
- Website: [getnora.dev](https://getnora.dev) - Website: [getnora.dev](https://getnora.dev)
- Telegram: [@getnora](https://t.me/getnora) - Telegram: [@DevITWay](https://t.me/DevITWay)
- GitHub: [@devitway](https://github.com/devitway) - GitHub: [@devitway](https://github.com/devitway)
- Email: devitway@gmail.com
## Contributing ## Contributing
@@ -269,6 +242,10 @@ NORA welcomes contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelin
## License ## License
MIT License see [LICENSE](LICENSE) MIT License - see [LICENSE](LICENSE)
Copyright (c) 2026 DevITWay Copyright (c) 2026 DevITWay
---
**🐿️ N○RA** - Organized like a chipmunk's stash | Built with Rust by [DevITWay](https://t.me/DevITWay)

View File

@@ -4,7 +4,6 @@
| Version | Supported | | Version | Supported |
| ------- | ------------------ | | ------- | ------------------ |
| 0.3.x | :white_check_mark: |
| 0.2.x | :white_check_mark: | | 0.2.x | :white_check_mark: |
| < 0.2 | :x: | | < 0.2 | :x: |
@@ -51,6 +50,4 @@ When deploying NORA:
## Acknowledgments ## Acknowledgments
We appreciate responsible disclosure and will acknowledge security researchers who report valid vulnerabilities in our release notes and CHANGELOG, unless the reporter requests anonymity. We appreciate responsible disclosure and will acknowledge security researchers who report valid vulnerabilities.
If you have previously reported a vulnerability and would like to be credited, please let us know.

View File

@@ -1,6 +0,0 @@
# Artifact Hub repository metadata
# https://artifacthub.io/docs/topics/repositories/
repositoryID: null # filled by Artifact Hub after registration
owners:
- name: DevITWay
email: devitway@gmail.com

View File

@@ -1,4 +0,0 @@
# NORA clippy configuration
cognitive-complexity-threshold = 25
too-many-arguments-threshold = 7
type-complexity-threshold = 300

View File

@@ -9,7 +9,6 @@ ignore = [
] ]
[licenses] [licenses]
unused-allowed-license = "allow"
# Allowed open-source licenses # Allowed open-source licenses
allow = [ allow = [
"MIT", "MIT",
@@ -25,7 +24,6 @@ allow = [
"Zlib", "Zlib",
"CDLA-Permissive-2.0", # webpki-roots (CA certificates bundle) "CDLA-Permissive-2.0", # webpki-roots (CA certificates bundle)
"MPL-2.0", "MPL-2.0",
"NCSA", # libfuzzer-sys (LLVM fuzzer)
] ]
[bans] [bans]

View File

@@ -1,33 +0,0 @@
# syntax=docker/dockerfile:1.4
# Binary is pre-built by CI (cargo build --release) and passed via context
# Runtime: scratch — compatible with Astra Linux SE (FSTEC certified)
# To switch to official base: replace FROM scratch with
# FROM registry.astralinux.ru/library/alse:latest
# RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/*
FROM alpine:3.20@sha256:a4f4213abb84c497377b8544c81b3564f313746700372ec4fe84653e4fb03805 AS certs
RUN apk add --no-cache ca-certificates \
&& addgroup -S -g 10001 nora && adduser -S -u 10001 -G nora nora
FROM scratch
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=certs /etc/passwd /etc/passwd
COPY --from=certs /etc/group /etc/group
COPY nora /usr/local/bin/nora
ENV RUST_LOG=info
ENV NORA_HOST=0.0.0.0
ENV NORA_PORT=4000
ENV NORA_STORAGE_MODE=local
ENV NORA_STORAGE_PATH=/data/storage
ENV NORA_AUTH_TOKEN_STORAGE=/data/tokens
EXPOSE 4000
VOLUME ["/data"]
USER nora
ENTRYPOINT ["/usr/local/bin/nora"]
CMD ["serve"]

View File

@@ -1,33 +0,0 @@
# syntax=docker/dockerfile:1.4
# Binary is pre-built by CI (cargo build --release) and passed via context
# Runtime: scratch — compatible with RED OS (FSTEC certified)
# To switch to official base: replace FROM scratch with
# FROM registry.red-soft.ru/redos/redos:8
# RUN dnf install -y ca-certificates && dnf clean all
FROM alpine:3.20@sha256:a4f4213abb84c497377b8544c81b3564f313746700372ec4fe84653e4fb03805 AS certs
RUN apk add --no-cache ca-certificates \
&& addgroup -S -g 10001 nora && adduser -S -u 10001 -G nora nora
FROM scratch
COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=certs /etc/passwd /etc/passwd
COPY --from=certs /etc/group /etc/group
COPY nora /usr/local/bin/nora
ENV RUST_LOG=info
ENV NORA_HOST=0.0.0.0
ENV NORA_PORT=4000
ENV NORA_STORAGE_MODE=local
ENV NORA_STORAGE_PATH=/data/storage
ENV NORA_AUTH_TOKEN_STORAGE=/data/tokens
EXPOSE 4000
VOLUME ["/data"]
USER nora
ENTRYPOINT ["/usr/local/bin/nora"]
CMD ["serve"]

83
deploy/demo-traffic.sh Normal file
View File

@@ -0,0 +1,83 @@
#!/bin/bash
# Demo traffic simulator for NORA registry
# Generates random registry activity for dashboard demo
REGISTRY="http://localhost:4000"
LOG_FILE="/var/log/nora-demo-traffic.log"
# Sample packages to fetch
NPM_PACKAGES=("lodash" "express" "react" "axios" "moment" "underscore" "chalk" "debug")
MAVEN_ARTIFACTS=(
"org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom"
"com/google/guava/guava/31.1-jre/guava-31.1-jre.pom"
"org/slf4j/slf4j-api/2.0.0/slf4j-api-2.0.0.pom"
)
DOCKER_IMAGES=("alpine" "busybox" "hello-world")
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" >> "$LOG_FILE"
}
# Random sleep between min and max seconds
random_sleep() {
local min=$1
local max=$2
local delay=$((RANDOM % (max - min + 1) + min))
sleep $delay
}
# Fetch random npm package
fetch_npm() {
local pkg=${NPM_PACKAGES[$RANDOM % ${#NPM_PACKAGES[@]}]}
log "NPM: fetching $pkg"
curl -s "$REGISTRY/npm/$pkg" > /dev/null 2>&1
}
# Fetch random maven artifact
fetch_maven() {
local artifact=${MAVEN_ARTIFACTS[$RANDOM % ${#MAVEN_ARTIFACTS[@]}]}
log "MAVEN: fetching $artifact"
curl -s "$REGISTRY/maven2/$artifact" > /dev/null 2>&1
}
# Docker push/pull cycle
docker_cycle() {
local img=${DOCKER_IMAGES[$RANDOM % ${#DOCKER_IMAGES[@]}]}
local tag="demo-$(date +%s)"
log "DOCKER: push/pull cycle for $img"
# Tag and push
docker tag "$img:latest" "localhost:4000/demo/$img:$tag" 2>/dev/null
docker push "localhost:4000/demo/$img:$tag" > /dev/null 2>&1
# Pull back
docker rmi "localhost:4000/demo/$img:$tag" > /dev/null 2>&1
docker pull "localhost:4000/demo/$img:$tag" > /dev/null 2>&1
# Cleanup
docker rmi "localhost:4000/demo/$img:$tag" > /dev/null 2>&1
}
# Main loop
log "Starting demo traffic simulator"
while true; do
# Random operation
op=$((RANDOM % 10))
case $op in
0|1|2|3) # 40% npm
fetch_npm
;;
4|5|6) # 30% maven
fetch_maven
;;
7|8|9) # 30% docker
docker_cycle
;;
esac
# Random delay: 30-120 seconds
random_sleep 30 120
done

View File

@@ -1,9 +1,12 @@
services: services:
nora: nora:
image: ghcr.io/getnora-io/nora:latest image: ghcr.io/getnora-io/nora:latest
build:
context: ..
dockerfile: Dockerfile
restart: unless-stopped restart: unless-stopped
expose: ports:
- "4000" - "4000:4000"
volumes: volumes:
- nora-data:/data - nora-data:/data
environment: environment:
@@ -11,28 +14,6 @@ services:
- NORA_HOST=0.0.0.0 - NORA_HOST=0.0.0.0
- NORA_PORT=4000 - NORA_PORT=4000
- NORA_AUTH_ENABLED=false - NORA_AUTH_ENABLED=false
healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://localhost:4000/health"]
interval: 10s
timeout: 5s
start_period: 5s
retries: 3
caddy:
image: caddy:2-alpine
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile:ro
- caddy-data:/data
- caddy-config:/config
depends_on:
nora:
condition: service_healthy
volumes: volumes:
nora-data: nora-data:
caddy-data:
caddy-config:

View File

@@ -0,0 +1,15 @@
[Unit]
Description=NORA Demo Traffic Simulator
After=docker.service
Requires=docker.service
[Service]
Type=simple
ExecStart=/opt/nora/demo-traffic.sh
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target

131
dist/install.sh vendored
View File

@@ -1,131 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# NORA Artifact Registry — install script
# Usage: curl -fsSL https://getnora.io/install.sh | bash
VERSION="${NORA_VERSION:-latest}"
ARCH=$(uname -m)
OS=$(uname -s | tr '[:upper:]' '[:lower:]')
INSTALL_DIR="/usr/local/bin"
CONFIG_DIR="/etc/nora"
DATA_DIR="/var/lib/nora"
LOG_DIR="/var/log/nora"
case "$ARCH" in
x86_64|amd64) ARCH="x86_64" ;;
aarch64|arm64) ARCH="aarch64" ;;
*) echo "Unsupported architecture: $ARCH"; exit 1 ;;
esac
echo "Installing NORA ($OS/$ARCH)..."
# Download binary
if [ "$VERSION" = "latest" ]; then
DOWNLOAD_URL="https://github.com/getnora-io/nora/releases/latest/download/nora-${OS}-${ARCH}"
else
DOWNLOAD_URL="https://github.com/getnora-io/nora/releases/download/${VERSION}/nora-${OS}-${ARCH}"
fi
echo "Downloading from $DOWNLOAD_URL..."
if command -v curl &>/dev/null; then
curl -fsSL -o /tmp/nora "$DOWNLOAD_URL"
elif command -v wget &>/dev/null; then
wget -qO /tmp/nora "$DOWNLOAD_URL"
else
echo "Error: curl or wget required"; exit 1
fi
chmod +x /tmp/nora
# Verify signature if cosign is available
if command -v cosign &>/dev/null; then
echo "Verifying binary signature..."
SIG_URL="${DOWNLOAD_URL}.sig"
CERT_URL="${DOWNLOAD_URL}.pem"
if curl -fsSL -o /tmp/nora.sig "$SIG_URL" 2>/dev/null && \
curl -fsSL -o /tmp/nora.pem "$CERT_URL" 2>/dev/null; then
cosign verify-blob --signature /tmp/nora.sig --certificate /tmp/nora.pem \
--certificate-identity-regexp "github.com/getnora-io/nora" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
/tmp/nora && echo "Signature verified." || echo "Warning: signature verification failed."
rm -f /tmp/nora.sig /tmp/nora.pem
else
echo "Signature files not available, skipping verification."
fi
else
echo "Install cosign for binary signature verification: https://docs.sigstore.dev/cosign/system_config/installation/"
fi
sudo mv /tmp/nora "$INSTALL_DIR/nora"
# Create system user
if ! id nora &>/dev/null; then
sudo useradd --system --shell /usr/sbin/nologin --home-dir "$DATA_DIR" --create-home nora
echo "Created system user: nora"
fi
# Create directories
sudo mkdir -p "$CONFIG_DIR" "$DATA_DIR" "$LOG_DIR"
sudo chown nora:nora "$DATA_DIR" "$LOG_DIR"
# Install default config if not exists
if [ ! -f "$CONFIG_DIR/nora.env" ]; then
cat > /tmp/nora.env << 'ENVEOF'
NORA_HOST=0.0.0.0
NORA_PORT=4000
NORA_STORAGE_PATH=/var/lib/nora
ENVEOF
sudo mv /tmp/nora.env "$CONFIG_DIR/nora.env"
sudo chmod 600 "$CONFIG_DIR/nora.env"
sudo chown nora:nora "$CONFIG_DIR/nora.env"
echo "Created default config: $CONFIG_DIR/nora.env"
fi
# Install systemd service
if [ -d /etc/systemd/system ]; then
cat > /tmp/nora.service << 'SVCEOF'
[Unit]
Description=NORA Artifact Registry
Documentation=https://getnora.dev
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=nora
Group=nora
ExecStart=/usr/local/bin/nora serve
WorkingDirectory=/etc/nora
Restart=on-failure
RestartSec=5
LimitNOFILE=65535
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/nora /var/log/nora
PrivateTmp=true
EnvironmentFile=-/etc/nora/nora.env
[Install]
WantedBy=multi-user.target
SVCEOF
sudo mv /tmp/nora.service /etc/systemd/system/nora.service
sudo systemctl daemon-reload
sudo systemctl enable nora
echo "Installed systemd service: nora"
fi
echo ""
echo "NORA installed successfully!"
echo ""
echo " Binary: $INSTALL_DIR/nora"
echo " Config: $CONFIG_DIR/nora.env"
echo " Data: $DATA_DIR"
echo " Version: $(nora --version 2>/dev/null || echo 'unknown')"
echo ""
echo "Next steps:"
echo " 1. Edit $CONFIG_DIR/nora.env"
echo " 2. sudo systemctl start nora"
echo " 3. curl http://localhost:4000/health"
echo ""

31
dist/nora.env.example vendored
View File

@@ -1,31 +0,0 @@
# NORA configuration — environment variables
# Copy to /etc/nora/nora.env and adjust
# Server
NORA_HOST=0.0.0.0
NORA_PORT=4000
# NORA_PUBLIC_URL=https://registry.example.com
# Storage
NORA_STORAGE_PATH=/var/lib/nora
# NORA_STORAGE_MODE=s3
# NORA_STORAGE_S3_URL=http://minio:9000
# NORA_STORAGE_BUCKET=registry
# Auth (optional)
# NORA_AUTH_ENABLED=true
# NORA_AUTH_HTPASSWD_FILE=/etc/nora/users.htpasswd
# Rate limiting
# NORA_RATE_LIMIT_ENABLED=true
# npm proxy
# NORA_NPM_PROXY=https://registry.npmjs.org
# NORA_NPM_METADATA_TTL=300
# NORA_NPM_PROXY_AUTH=user:pass
# PyPI proxy
# NORA_PYPI_PROXY=https://pypi.org/simple/
# Docker upstreams
# NORA_DOCKER_UPSTREAMS=https://registry-1.docker.io

28
dist/nora.service vendored
View File

@@ -1,28 +0,0 @@
[Unit]
Description=NORA Artifact Registry
Documentation=https://getnora.dev
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=nora
Group=nora
ExecStart=/usr/local/bin/nora serve
WorkingDirectory=/etc/nora
Restart=on-failure
RestartSec=5
LimitNOFILE=65535
# Security hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/nora /var/log/nora
PrivateTmp=true
# Environment
EnvironmentFile=-/etc/nora/nora.env
[Install]
WantedBy=multi-user.target

View File

@@ -1,5 +1,6 @@
services: services:
nora: nora:
build: .
image: ghcr.io/getnora-io/nora:latest image: ghcr.io/getnora-io/nora:latest
ports: ports:
- "4000:4000" - "4000:4000"

View File

@@ -1,13 +0,0 @@
# Документация NORA для Росреестра
## Структура
- `ТУ.md` — Технические условия
- `Руководство.md` — Руководство пользователя
- `Руководство_администратора.md` — Руководство администратора
- `SBOM.md` — Перечень компонентов (Software Bill of Materials)
## Статус
Подготовка документации для включения в Единый реестр российских программ
для электронных вычислительных машин и баз данных (Минцифры РФ).

View File

@@ -1,301 +0,0 @@
# Руководство администратора NORA
**Версия:** 0.2.32
**Дата:** 2026-03-16
**Правообладатель:** ООО «ТАИАРС» (торговая марка АРТАИС)
---
## 1. Общие сведения
NORA — многопротокольный реестр артефактов, предназначенный для хранения, кэширования и распространения программных компонентов. Программа обеспечивает централизованное управление зависимостями при разработке и сборке программного обеспечения.
### 1.1. Назначение
- Хранение и раздача артефактов по протоколам Docker (OCI), npm, Maven, PyPI, Cargo, Helm OCI и Raw.
- Проксирование и кэширование внешних репозиториев для ускорения сборок и обеспечения доступности при отсутствии соединения с сетью Интернет.
- Контроль целостности артефактов посредством верификации SHA-256.
- Аудит и протоколирование всех операций.
### 1.2. Системные требования
| Параметр | Минимальные | Рекомендуемые |
|----------|-------------|---------------|
| ОС | Linux (amd64, arm64) | Ubuntu 22.04+, RHEL 8+ |
| ЦПУ | 1 ядро | 2+ ядра |
| ОЗУ | 256 МБ | 1+ ГБ |
| Диск | 1 ГБ | 50+ ГБ (зависит от объёма хранимых артефактов) |
| Сеть | TCP-порт (по умолчанию 4000) | — |
### 1.3. Зависимости
Программа поставляется как единый статически слинкованный исполняемый файл. Внешние зависимости отсутствуют. Перечень библиотек, включённых в состав программы, приведён в файле `nora.cdx.json` (формат CycloneDX).
---
## 2. Установка
### 2.1. Автоматическая установка
```bash
curl -fsSL https://getnora.io/install.sh | bash
```
Скрипт выполняет следующие действия:
1. Определяет архитектуру процессора (amd64 или arm64).
2. Загружает исполняемый файл с GitHub Releases.
3. Создаёт системного пользователя `nora`.
4. Создаёт каталоги: `/etc/nora/`, `/var/lib/nora/`, `/var/log/nora/`.
5. Устанавливает файл конфигурации `/etc/nora/nora.env`.
6. Устанавливает и активирует systemd-сервис.
### 2.2. Ручная установка
```bash
# Загрузка
wget https://github.com/getnora-io/nora/releases/download/v1.0.0/nora-linux-x86_64
chmod +x nora-linux-x86_64
mv nora-linux-x86_64 /usr/local/bin/nora
# Создание пользователя
useradd --system --shell /usr/sbin/nologin --home-dir /var/lib/nora --create-home nora
# Создание каталогов
mkdir -p /etc/nora /var/lib/nora /var/log/nora
chown nora:nora /var/lib/nora /var/log/nora
# Установка systemd-сервиса
cp dist/nora.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable nora
```
### 2.3. Установка из Docker-образа
```bash
docker run -d \
--name nora \
-p 4000:4000 \
-v nora-data:/data \
ghcr.io/getnora-io/nora:latest
```
---
## 3. Конфигурация
Конфигурация задаётся через переменные окружения, файл `config.toml` или их комбинацию. Приоритет: переменные окружения > config.toml > значения по умолчанию.
### 3.1. Основные параметры
| Переменная | Описание | По умолчанию |
|-----------|----------|--------------|
| `NORA_HOST` | Адрес привязки | `127.0.0.1` |
| `NORA_PORT` | Порт | `4000` |
| `NORA_PUBLIC_URL` | Внешний URL (для генерации ссылок) | — |
| `NORA_STORAGE_PATH` | Путь к каталогу хранилища | `data/storage` |
| `NORA_STORAGE_MODE` | Тип хранилища: `local` или `s3` | `local` |
| `NORA_BODY_LIMIT_MB` | Максимальный размер тела запроса (МБ) | `2048` |
### 3.2. Аутентификация
| Переменная | Описание | По умолчанию |
|-----------|----------|--------------|
| `NORA_AUTH_ENABLED` | Включить аутентификацию | `false` |
| `NORA_AUTH_HTPASSWD_FILE` | Путь к файлу htpasswd | `users.htpasswd` |
Создание пользователя:
```bash
htpasswd -Bc /etc/nora/users.htpasswd admin
```
Роли: `admin` (полный доступ), `write` (чтение и запись), `read` (только чтение, по умолчанию).
### 3.3. Проксирование внешних репозиториев
| Переменная | Описание | По умолчанию |
|-----------|----------|--------------|
| `NORA_NPM_PROXY` | URL npm-реестра | `https://registry.npmjs.org` |
| `NORA_NPM_PROXY_AUTH` | Учётные данные (`user:pass`) | — |
| `NORA_NPM_METADATA_TTL` | TTL кэша метаданных (секунды) | `300` |
| `NORA_PYPI_PROXY` | URL PyPI-реестра | `https://pypi.org/simple/` |
| `NORA_MAVEN_PROXIES` | Список Maven-репозиториев через запятую | `https://repo1.maven.org/maven2` |
| `NORA_DOCKER_UPSTREAMS` | Docker-реестры, формат: `url\|auth,url2` | `https://registry-1.docker.io` |
### 3.4. Ограничение частоты запросов
| Переменная | Описание | По умолчанию |
|-----------|----------|--------------|
| `NORA_RATE_LIMIT_ENABLED` | Включить ограничение | `true` |
| `NORA_RATE_LIMIT_GENERAL_RPS` | Запросов в секунду (общие) | `100` |
| `NORA_RATE_LIMIT_AUTH_RPS` | Запросов в секунду (аутентификация) | `1` |
| `NORA_RATE_LIMIT_UPLOAD_RPS` | Запросов в секунду (загрузка) | `200` |
---
## 4. Управление сервисом
### 4.1. Запуск и остановка
```bash
systemctl start nora # Запуск
systemctl stop nora # Остановка
systemctl restart nora # Перезапуск
systemctl status nora # Статус
journalctl -u nora -f # Просмотр журнала
```
### 4.2. Проверка работоспособности
```bash
curl http://localhost:4000/health
```
Ответ при нормальной работе:
```json
{
"status": "healthy",
"version": "1.0.0",
"storage": { "backend": "local", "reachable": true },
"registries": { "docker": "ok", "npm": "ok", "maven": "ok", "cargo": "ok", "pypi": "ok" }
}
```
### 4.3. Метрики (Prometheus)
```
GET /metrics
```
Экспортируются: количество запросов, латентность, загрузки и выгрузки по протоколам.
---
## 5. Резервное копирование и восстановление
### 5.1. Создание резервной копии
```bash
nora backup --output /backup/nora-$(date +%Y%m%d).tar.gz
```
### 5.2. Восстановление
```bash
nora restore --input /backup/nora-20260316.tar.gz
```
### 5.3. Сборка мусора
```bash
nora gc --dry-run # Просмотр (без удаления)
nora gc # Удаление осиротевших блобов
```
---
## 6. Предварительное кэширование (nora mirror)
Команда `nora mirror` позволяет заранее загрузить зависимости через прокси-кэш NORA. Это обеспечивает доступность артефактов при работе в изолированных средах без доступа к сети Интернет.
### 6.1. Кэширование по lockfile
```bash
nora mirror npm --lockfile package-lock.json --registry http://localhost:4000
nora mirror pip --lockfile requirements.txt --registry http://localhost:4000
nora mirror cargo --lockfile Cargo.lock --registry http://localhost:4000
```
### 6.2. Кэширование по списку пакетов
```bash
nora mirror npm --packages lodash,express --registry http://localhost:4000
nora mirror npm --packages lodash --all-versions --registry http://localhost:4000
```
### 6.3. Параметры
| Флаг | Описание | По умолчанию |
|------|----------|--------------|
| `--registry` | URL экземпляра NORA | `http://localhost:4000` |
| `--concurrency` | Количество параллельных загрузок | `8` |
| `--all-versions` | Загрузить все версии (только с `--packages`) | — |
---
## 7. Миграция хранилища
Перенос артефактов между локальным хранилищем и S3:
```bash
nora migrate --from local --to s3 --dry-run # Просмотр
nora migrate --from local --to s3 # Выполнение
```
---
## 8. Безопасность
### 8.1. Контроль целостности
При проксировании npm-пакетов NORA вычисляет и сохраняет контрольную сумму SHA-256 для каждого тарбола. При повторной выдаче из кэша контрольная сумма проверяется. В случае расхождения запрос отклоняется, а в журнал записывается предупреждение уровня SECURITY.
### 8.2. Защита от подмены пакетов
- Валидация имён файлов при публикации (защита от обхода каталогов).
- Проверка соответствия имени пакета в URL и теле запроса.
- Иммутабельность версий: повторная публикация той же версии запрещена.
### 8.3. Аудит
Все операции (загрузка, выгрузка, обращения к кэшу, ошибки) фиксируются в файле `audit.jsonl` в каталоге хранилища. Формат — JSON Lines, одна запись на строку.
### 8.4. Усиление systemd
Файл сервиса содержит параметры безопасности:
- `NoNewPrivileges=true` — запрет повышения привилегий.
- `ProtectSystem=strict` — файловая система только для чтения, кроме указанных каталогов.
- `ProtectHome=true` — запрет доступа к домашним каталогам.
- `PrivateTmp=true` — изолированный каталог временных файлов.
---
## 9. Точки подключения (endpoints)
| Протокол | Endpoint | Описание |
|----------|----------|----------|
| Docker / OCI | `/v2/` | Docker Registry V2 API |
| npm | `/npm/` | npm-реестр (прокси + публикация) |
| Maven | `/maven2/` | Maven-репозиторий |
| PyPI | `/simple/` | Python Simple API (PEP 503) |
| Cargo | `/cargo/` | Cargo-реестр |
| Helm | `/v2/` (OCI) | Helm-чарты через OCI-протокол |
| Raw | `/raw/` | Произвольные файлы |
| Мониторинг | `/health`, `/ready`, `/metrics` | Проверка и метрики |
| Интерфейс | `/ui/` | Веб-интерфейс управления |
| Документация API | `/api-docs` | OpenAPI (Swagger UI) |
---
## 10. Устранение неполадок
### Сервис не запускается
```bash
journalctl -u nora --no-pager -n 50
```
Частые причины: занят порт, недоступен каталог хранилища, ошибка в конфигурации.
### Прокси-кэш не работает
1. Проверьте доступность внешнего реестра: `curl https://registry.npmjs.org/lodash`.
2. Убедитесь, что переменная `NORA_NPM_PROXY` задана корректно.
3. При использовании приватного реестра укажите `NORA_NPM_PROXY_AUTH`.
### Ошибка целостности (Integrity check failed)
Контрольная сумма кэшированного тарбола не совпадает с сохранённой. Возможные причины: повреждение файловой системы или несанкционированное изменение файла. Удалите повреждённый файл из каталога хранилища — NORA загрузит его заново из внешнего реестра.

View File

@@ -1,165 +0,0 @@
# Технические условия
## Программа «NORA — Реестр артефактов»
**Версия документа:** 0.2.32
**Дата:** 2026-03-16
**Правообладатель:** ООО «ТАИАРС» (торговая марка АРТАИС)
---
## 1. Наименование и обозначение
**Полное наименование:** NORA — многопротокольный реестр артефактов.
**Краткое наименование:** NORA.
**Обозначение:** nora-registry.
---
## 2. Назначение
Программа предназначена для хранения, кэширования и распространения программных компонентов (артефактов), используемых при разработке, сборке и развёртывании программного обеспечения.
### 2.1. Область применения
- Организация внутренних репозиториев программных компонентов.
- Проксирование и кэширование общедоступных репозиториев (npmjs.org, PyPI, Maven Central, Docker Hub, crates.io).
- Обеспечение доступности зависимостей в изолированных средах без доступа к сети Интернет (air-gapped).
- Контроль целостности и безопасности цепочки поставки программного обеспечения.
### 2.2. Класс программного обеспечения
Инструментальное программное обеспечение для разработки и DevOps.
Код ОКПД2: 62.01 — Разработка компьютерного программного обеспечения.
---
## 3. Функциональные характеристики
### 3.1. Поддерживаемые протоколы
| Протокол | Стандарт | Назначение |
|----------|----------|------------|
| Docker / OCI | OCI Distribution Spec v1.0 | Контейнерные образы, Helm-чарты |
| npm | npm Registry API | Библиотеки JavaScript / TypeScript |
| Maven | Maven Repository Layout | Библиотеки Java / Kotlin |
| PyPI | PEP 503 (Simple API) | Библиотеки Python |
| Cargo | Cargo Registry Protocol | Библиотеки Rust |
| Raw | HTTP PUT/GET | Произвольные файлы |
### 3.2. Режимы работы
1. **Хранилище (hosted):** приём и хранение артефактов, опубликованных пользователями.
2. **Прокси-кэш (proxy):** прозрачное проксирование запросов к внешним репозиториям с локальным кэшированием.
3. **Комбинированный:** одновременная работа в режимах хранилища и прокси-кэша (поиск сначала в локальном хранилище, затем во внешнем репозитории).
### 3.3. Управление доступом
- Аутентификация на основе htpasswd (bcrypt).
- Ролевая модель: `read` (чтение), `write` (чтение и запись), `admin` (полный доступ).
- Токены доступа с ограниченным сроком действия.
### 3.4. Безопасность
- Контроль целостности кэшированных артефактов (SHA-256).
- Защита от обхода каталогов (path traversal) при публикации.
- Проверка соответствия имени пакета в URL и теле запроса.
- Иммутабельность опубликованных версий.
- Аудит всех операций в формате JSON Lines.
- Поддержка TLS при размещении за обратным прокси-сервером.
### 3.5. Эксплуатация
- Предварительное кэширование зависимостей (`nora mirror`) по файлам фиксации версий (lockfile).
- Сборка мусора (`nora gc`) — удаление осиротевших блобов.
- Резервное копирование и восстановление (`nora backup`, `nora restore`).
- Миграция между локальным хранилищем и S3-совместимым объектным хранилищем.
- Мониторинг: эндпоинты `/health`, `/ready`, `/metrics` (формат Prometheus).
- Веб-интерфейс для просмотра содержимого реестра.
- Документация API в формате OpenAPI 3.0.
---
## 4. Технические характеристики
### 4.1. Среда исполнения
| Параметр | Значение |
|----------|----------|
| Язык реализации | Rust |
| Формат поставки | Единый исполняемый файл (статическая линковка) |
| Поддерживаемые ОС | Linux (ядро 4.15+) |
| Архитектуры | x86_64 (amd64), aarch64 (arm64) |
| Контейнеризация | Docker-образ на базе `scratch` |
| Системная интеграция | systemd (файл сервиса в комплекте) |
### 4.2. Хранение данных
| Параметр | Значение |
|----------|----------|
| Локальное хранилище | Файловая система (ext4, XFS, ZFS) |
| Объектное хранилище | S3-совместимое API (MinIO, Yandex Object Storage, Selectel S3) |
| Структура | Иерархическая: `{protocol}/{package}/{artifact}` |
| Аудит | Append-only JSONL файл |
### 4.3. Конфигурация
| Источник | Приоритет |
|----------|-----------|
| Переменные окружения (`NORA_*`) | Высший |
| Файл `config.toml` | Средний |
| Значения по умолчанию | Низший |
### 4.4. Производительность
| Параметр | Значение |
|----------|----------|
| Время запуска | < 100 мс |
| Обслуживание из кэша | < 2 мс (метаданные), < 10 мс (артефакты до 1 МБ) |
| Параллельная обработка | Асинхронный ввод-вывод (tokio runtime) |
| Ограничение частоты | Настраиваемое (по умолчанию 100 запросов/сек) |
---
## 5. Лицензирование
| Компонент | Лицензия |
|-----------|----------|
| NORA (core) | MIT License |
| NORA Enterprise | Проприетарная |
Полный перечень лицензий включённых библиотек приведён в файле SBOM (формат CycloneDX).
---
## 6. Комплектность
| Компонент | Описание |
|-----------|----------|
| `nora` | Исполняемый файл |
| `nora.service` | Файл systemd-сервиса |
| `nora.env.example` | Шаблон конфигурации |
| `install.sh` | Скрипт установки |
| `nora.cdx.json` | SBOM в формате CycloneDX |
| Руководство администратора | Настоящий комплект документации |
| Руководство пользователя | Настоящий комплект документации |
| Технические условия | Настоящий документ |
---
## 7. Контактная информация
**Правообладатель:** ООО «ТАИАРС»
**Торговая марка:** АРТАИС
**Сайт продукта:** https://getnora.io
**Документация:** https://getnora.dev
**Исходный код:** https://github.com/getnora-io/nora
**Поддержка:** https://t.me/getnora

View File

@@ -1,221 +0,0 @@
# Руководство пользователя NORA
**Версия:** 0.2.32
**Дата:** 2026-03-16
**Правообладатель:** ООО «ТАИАРС» (торговая марка АРТАИС)
---
## 1. Общие сведения
NORA — реестр артефактов для команд разработки. Программа обеспечивает хранение и кэширование библиотек, Docker-образов и иных программных компонентов, используемых при сборке приложений.
Данное руководство предназначено для разработчиков, которые используют NORA в качестве источника зависимостей.
---
## 2. Настройка рабочего окружения
### 2.1. npm / Node.js
Укажите NORA в качестве реестра:
```bash
npm config set registry http://nora.example.com:4000/npm
```
Или создайте файл `.npmrc` в корне проекта:
```
registry=http://nora.example.com:4000/npm
```
После этого все команды `npm install` будут загружать пакеты через NORA. При первом обращении NORA загрузит пакет из внешнего реестра (npmjs.org) и сохранит его в кэш. Последующие обращения обслуживаются из кэша.
### 2.2. Docker
```bash
docker login nora.example.com:4000
docker pull nora.example.com:4000/library/nginx:latest
docker push nora.example.com:4000/myteam/myapp:1.0.0
```
### 2.3. Maven
Добавьте репозиторий в `settings.xml`:
```xml
<mirrors>
<mirror>
<id>nora</id>
<mirrorOf>central</mirrorOf>
<url>http://nora.example.com:4000/maven2</url>
</mirror>
</mirrors>
```
### 2.4. Python / pip
```bash
pip install --index-url http://nora.example.com:4000/simple flask
```
Или в `pip.conf`:
```ini
[global]
index-url = http://nora.example.com:4000/simple
```
### 2.5. Cargo / Rust
Настройка в `~/.cargo/config.toml`:
```toml
[registries.nora]
index = "sparse+http://nora.example.com:4000/cargo/"
[source.crates-io]
replace-with = "nora"
```
### 2.6. Helm
Helm использует OCI-протокол через Docker Registry API:
```bash
helm push mychart-0.1.0.tgz oci://nora.example.com:4000/helm
helm pull oci://nora.example.com:4000/helm/mychart --version 0.1.0
```
---
## 3. Публикация пакетов
### 3.1. npm
```bash
npm publish --registry http://nora.example.com:4000/npm
```
Требования:
- Файл `package.json` с полями `name` и `version`.
- Каждая версия публикуется однократно. Повторная публикация той же версии запрещена.
### 3.2. Docker
```bash
docker tag myapp:latest nora.example.com:4000/myteam/myapp:1.0.0
docker push nora.example.com:4000/myteam/myapp:1.0.0
```
### 3.3. Maven
```bash
mvn deploy -DaltDeploymentRepository=nora::default::http://nora.example.com:4000/maven2
```
### 3.4. Raw (произвольные файлы)
```bash
# Загрузка
curl -X PUT --data-binary @release.tar.gz http://nora.example.com:4000/raw/builds/release-1.0.tar.gz
# Скачивание
curl -O http://nora.example.com:4000/raw/builds/release-1.0.tar.gz
```
---
## 4. Работа в изолированной среде
Если сборочный сервер не имеет доступа к сети Интернет, используйте предварительное кэширование.
### 4.1. Кэширование зависимостей проекта
На машине с доступом к Интернету и NORA выполните:
```bash
nora mirror npm --lockfile package-lock.json --registry http://nora.example.com:4000
```
После этого все зависимости из lockfile будут доступны через NORA, даже если связь с внешними реестрами отсутствует.
### 4.2. Кэширование всех версий пакета
```bash
nora mirror npm --packages lodash,express --all-versions --registry http://nora.example.com:4000
```
Эта команда загрузит все опубликованные версии указанных пакетов.
---
## 5. Веб-интерфейс
NORA предоставляет веб-интерфейс для просмотра содержимого реестра:
```
http://nora.example.com:4000/ui/
```
Доступные функции:
- Просмотр списка артефактов по протоколам.
- Количество версий и размер каждого пакета.
- Журнал последних операций.
- Метрики загрузок.
---
## 6. Документация API
Интерактивная документация API доступна по адресу:
```
http://nora.example.com:4000/api-docs
```
Формат: OpenAPI 3.0 (Swagger UI).
---
## 7. Аутентификация
Если администратор включил аутентификацию, для операций записи требуется токен.
### 7.1. Получение токена
```bash
curl -u admin:password http://nora.example.com:4000/auth/token
```
### 7.2. Использование токена
```bash
# npm
npm config set //nora.example.com:4000/npm/:_authToken TOKEN
# Docker
docker login nora.example.com:4000
# curl
curl -H "Authorization: Bearer TOKEN" http://nora.example.com:4000/npm/my-package
```
Операции чтения по умолчанию не требуют аутентификации (роль `read` назначается автоматически).
---
## 8. Часто задаваемые вопросы
**В: Что произойдёт, если внешний реестр (npmjs.org) станет недоступен?**
О: NORA продолжит обслуживать запросы из кэша. Пакеты, которые ранее не запрашивались, будут недоступны до восстановления связи. Для предотвращения такой ситуации используйте `nora mirror`.
**В: Можно ли публиковать приватные пакеты?**
О: Да. Пакеты, опубликованные через `npm publish` или `docker push`, сохраняются в локальном хранилище NORA и доступны всем пользователям данного экземпляра.
**В: Как обновить кэш метаданных?**
О: Кэш метаданных npm обновляется автоматически по истечении TTL (по умолчанию 5 минут). Для немедленного обновления удалите файл `metadata.json` из каталога хранилища.
**В: Поддерживаются ли scoped-пакеты npm (@scope/package)?**
О: Да, полностью. Например: `npm install @babel/core --registry http://nora.example.com:4000/npm`.

View File

@@ -1,23 +0,0 @@
[package]
name = "nora-fuzz"
version = "0.0.0"
publish = false
license = "MIT"
edition = "2021"
[package.metadata]
cargo-fuzz = true
[dependencies]
libfuzzer-sys = "0.4"
nora-registry = { path = "../nora-registry" }
[[bin]]
name = "fuzz_validation"
path = "fuzz_targets/fuzz_validation.rs"
doc = false
[[bin]]
name = "fuzz_docker_manifest"
path = "fuzz_targets/fuzz_docker_manifest.rs"
doc = false

View File

@@ -1,8 +0,0 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use nora_registry::docker_fuzz::detect_manifest_media_type;
fuzz_target!(|data: &[u8]| {
// Fuzz Docker manifest parser — must never panic on any input
let _ = detect_manifest_media_type(data);
});

View File

@@ -1,13 +0,0 @@
#![no_main]
use libfuzzer_sys::fuzz_target;
use nora_registry::validation::{
validate_digest, validate_docker_name, validate_docker_reference, validate_storage_key,
};
fuzz_target!(|data: &str| {
// Fuzz all validators — they must never panic on any input
let _ = validate_storage_key(data);
let _ = validate_docker_name(data);
let _ = validate_digest(data);
let _ = validate_docker_reference(data);
});

98
install.sh Normal file
View File

@@ -0,0 +1,98 @@
#!/usr/bin/env sh
# NORA installer — https://getnora.io/install.sh
# Usage: curl -fsSL https://getnora.io/install.sh | sh
set -e
REPO="getnora-io/nora"
BINARY="nora"
INSTALL_DIR="/usr/local/bin"
# ── Detect OS and architecture ──────────────────────────────────────────────
OS="$(uname -s)"
ARCH="$(uname -m)"
case "$OS" in
Linux) os="linux" ;;
Darwin) os="darwin" ;;
*)
echo "Unsupported OS: $OS"
echo "Please download manually: https://github.com/$REPO/releases/latest"
exit 1
;;
esac
case "$ARCH" in
x86_64 | amd64) arch="amd64" ;;
aarch64 | arm64) arch="arm64" ;;
*)
echo "Unsupported architecture: $ARCH"
echo "Please download manually: https://github.com/$REPO/releases/latest"
exit 1
;;
esac
ASSET="${BINARY}-${os}-${arch}"
# ── Get latest release version ──────────────────────────────────────────────
VERSION="$(curl -fsSL "https://api.github.com/repos/$REPO/releases/latest" \
| grep '"tag_name"' \
| sed 's/.*"tag_name": *"\([^"]*\)".*/\1/')"
if [ -z "$VERSION" ]; then
echo "Failed to get latest version"
exit 1
fi
echo "Installing NORA $VERSION ($os/$arch)..."
# ── Download binary and checksum ────────────────────────────────────────────
BASE_URL="https://github.com/$REPO/releases/download/$VERSION"
TMP_DIR="$(mktemp -d)"
trap 'rm -rf "$TMP_DIR"' EXIT
echo "Downloading $ASSET..."
curl -fsSL "$BASE_URL/$ASSET" -o "$TMP_DIR/$BINARY"
curl -fsSL "$BASE_URL/$ASSET.sha256" -o "$TMP_DIR/$ASSET.sha256"
# ── Verify checksum ─────────────────────────────────────────────────────────
echo "Verifying checksum..."
EXPECTED="$(awk '{print $1}' "$TMP_DIR/$ASSET.sha256")"
ACTUAL="$(sha256sum "$TMP_DIR/$BINARY" | awk '{print $1}')"
if [ "$EXPECTED" != "$ACTUAL" ]; then
echo "Checksum mismatch!"
echo " Expected: $EXPECTED"
echo " Actual: $ACTUAL"
exit 1
fi
echo "Checksum OK"
# ── Install ─────────────────────────────────────────────────────────────────
chmod +x "$TMP_DIR/$BINARY"
if [ -w "$INSTALL_DIR" ]; then
mv "$TMP_DIR/$BINARY" "$INSTALL_DIR/$BINARY"
elif command -v sudo >/dev/null 2>&1; then
sudo mv "$TMP_DIR/$BINARY" "$INSTALL_DIR/$BINARY"
else
# Fallback to ~/.local/bin
INSTALL_DIR="$HOME/.local/bin"
mkdir -p "$INSTALL_DIR"
mv "$TMP_DIR/$BINARY" "$INSTALL_DIR/$BINARY"
echo "Installed to $INSTALL_DIR/$BINARY"
echo "Make sure $INSTALL_DIR is in your PATH"
fi
# ── Done ────────────────────────────────────────────────────────────────────
echo ""
echo "NORA $VERSION installed to $INSTALL_DIR/$BINARY"
echo ""
nora --version 2>/dev/null || true

BIN
logo.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.9 KiB

23
nora-cli/Cargo.toml Normal file
View File

@@ -0,0 +1,23 @@
[package]
name = "nora-cli"
version.workspace = true
edition.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
homepage.workspace = true
description = "CLI tool for NORA registry"
[[bin]]
name = "nora-cli"
path = "src/main.rs"
[dependencies]
tokio.workspace = true
reqwest.workspace = true
serde.workspace = true
serde_json.workspace = true
clap = { version = "4", features = ["derive"] }
indicatif = "0.18"
tar = "0.4"
flate2 = "1.1"

55
nora-cli/src/main.rs Normal file
View File

@@ -0,0 +1,55 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT
use clap::{Parser, Subcommand};
#[derive(Parser)]
#[command(name = "nora-cli")]
#[command(about = "CLI tool for Nora registry")]
struct Cli {
#[command(subcommand)]
command: Commands,
}
#[derive(Subcommand)]
enum Commands {
/// Login to a registry
Login {
#[arg(long)]
registry: String,
#[arg(short, long)]
username: String,
},
/// Push an artifact
Push {
#[arg(long)]
registry: String,
path: String,
},
/// Pull an artifact
Pull {
#[arg(long)]
registry: String,
artifact: String,
},
}
#[tokio::main]
async fn main() {
let cli = Cli::parse();
match cli.command {
Commands::Login { registry, username } => {
println!("Logging in to {} as {}", registry, username);
// TODO: implement
}
Commands::Push { registry, path } => {
println!("Pushing {} to {}", path, registry);
// TODO: implement
}
Commands::Pull { registry, artifact } => {
println!("Pulling {} from {}", artifact, registry);
// TODO: implement
}
}
}

View File

@@ -10,10 +10,6 @@ description = "Cloud-Native Artifact Registry - Fast, lightweight, multi-protoco
keywords = ["registry", "docker", "artifacts", "cloud-native", "devops"] keywords = ["registry", "docker", "artifacts", "cloud-native", "devops"]
categories = ["command-line-utilities", "development-tools", "web-programming"] categories = ["command-line-utilities", "development-tools", "web-programming"]
[lib]
name = "nora_registry"
path = "src/lib.rs"
[[bin]] [[bin]]
name = "nora" name = "nora"
path = "src/main.rs" path = "src/main.rs"
@@ -30,7 +26,7 @@ sha2.workspace = true
async-trait.workspace = true async-trait.workspace = true
hmac.workspace = true hmac.workspace = true
hex.workspace = true hex.workspace = true
toml = "1.1" toml = "1.0"
uuid = { version = "1", features = ["v4"] } uuid = { version = "1", features = ["v4"] }
bcrypt = "0.19" bcrypt = "0.19"
base64 = "0.22" base64 = "0.22"
@@ -49,21 +45,7 @@ tower_governor = "0.8"
governor = "0.10" governor = "0.10"
parking_lot = "0.12" parking_lot = "0.12"
zeroize = { version = "1.8", features = ["derive"] } zeroize = { version = "1.8", features = ["derive"] }
argon2 = { version = "0.5", features = ["std", "rand"] }
tower-http = { version = "0.6", features = ["set-header"] }
percent-encoding = "2"
[dev-dependencies] [dev-dependencies]
proptest = "1"
tempfile = "3" tempfile = "3"
wiremock = "0.6" wiremock = "0.6"
criterion = { version = "0.8", features = ["html_reports"] }
tower = { version = "0.5", features = ["util"] }
http-body-util = "0.1"
[[bench]]
name = "parsing"
harness = false
[lints]
workspace = true

View File

@@ -1,109 +0,0 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use nora_registry::validation::{
validate_digest, validate_docker_name, validate_docker_reference, validate_storage_key,
};
fn bench_validation(c: &mut Criterion) {
let mut group = c.benchmark_group("validation");
group.bench_function("storage_key_short", |b| {
b.iter(|| validate_storage_key(black_box("docker/alpine/blobs/sha256:abc123")))
});
group.bench_function("storage_key_long", |b| {
let key = "maven/com/example/deep/nested/path/artifact-1.0.0-SNAPSHOT.jar";
b.iter(|| validate_storage_key(black_box(key)))
});
group.bench_function("storage_key_reject", |b| {
b.iter(|| validate_storage_key(black_box("../etc/passwd")))
});
group.bench_function("docker_name_simple", |b| {
b.iter(|| validate_docker_name(black_box("library/alpine")))
});
group.bench_function("docker_name_nested", |b| {
b.iter(|| validate_docker_name(black_box("my-org/sub/repo-name")))
});
group.bench_function("docker_name_reject", |b| {
b.iter(|| validate_docker_name(black_box("INVALID/NAME")))
});
group.bench_function("digest_sha256", |b| {
b.iter(|| {
validate_digest(black_box(
"sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
))
})
});
group.bench_function("digest_reject", |b| {
b.iter(|| validate_digest(black_box("md5:abc")))
});
group.bench_function("reference_tag", |b| {
b.iter(|| validate_docker_reference(black_box("v1.2.3-alpine")))
});
group.bench_function("reference_digest", |b| {
b.iter(|| {
validate_docker_reference(black_box(
"sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
))
})
});
group.finish();
}
fn bench_manifest_detection(c: &mut Criterion) {
let mut group = c.benchmark_group("manifest_detection");
let docker_v2 = serde_json::json!({
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"schemaVersion": 2,
"config": {"mediaType": "application/vnd.docker.container.image.v1+json", "digest": "sha256:abc"},
"layers": [{"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "digest": "sha256:def", "size": 1000}]
})
.to_string();
let oci_index = serde_json::json!({
"schemaVersion": 2,
"manifests": [
{"digest": "sha256:aaa", "platform": {"os": "linux", "architecture": "amd64"}},
{"digest": "sha256:bbb", "platform": {"os": "linux", "architecture": "arm64"}}
]
})
.to_string();
let minimal = serde_json::json!({"schemaVersion": 2}).to_string();
group.bench_function("docker_v2_explicit", |b| {
b.iter(|| {
nora_registry::docker_fuzz::detect_manifest_media_type(black_box(docker_v2.as_bytes()))
})
});
group.bench_function("oci_index", |b| {
b.iter(|| {
nora_registry::docker_fuzz::detect_manifest_media_type(black_box(oci_index.as_bytes()))
})
});
group.bench_function("minimal_json", |b| {
b.iter(|| {
nora_registry::docker_fuzz::detect_manifest_media_type(black_box(minimal.as_bytes()))
})
});
group.bench_function("invalid_json", |b| {
b.iter(|| nora_registry::docker_fuzz::detect_manifest_media_type(black_box(b"not json")))
});
group.finish();
}
criterion_group!(benches, bench_validation, bench_manifest_detection);
criterion_main!(benches);

View File

@@ -99,139 +99,3 @@ impl Default for ActivityLog {
Self::new(50) Self::new(50)
} }
} }
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_action_type_display() {
assert_eq!(ActionType::Pull.to_string(), "PULL");
assert_eq!(ActionType::Push.to_string(), "PUSH");
assert_eq!(ActionType::CacheHit.to_string(), "CACHE");
assert_eq!(ActionType::ProxyFetch.to_string(), "PROXY");
}
#[test]
fn test_action_type_equality() {
assert_eq!(ActionType::Pull, ActionType::Pull);
assert_ne!(ActionType::Pull, ActionType::Push);
}
#[test]
fn test_activity_entry_new() {
let entry = ActivityEntry::new(
ActionType::Pull,
"nginx:latest".to_string(),
"docker",
"LOCAL",
);
assert_eq!(entry.action, ActionType::Pull);
assert_eq!(entry.artifact, "nginx:latest");
assert_eq!(entry.registry, "docker");
assert_eq!(entry.source, "LOCAL");
}
#[test]
fn test_activity_log_push_and_len() {
let log = ActivityLog::new(10);
assert!(log.is_empty());
assert_eq!(log.len(), 0);
log.push(ActivityEntry::new(
ActionType::Push,
"test:v1".to_string(),
"docker",
"LOCAL",
));
assert!(!log.is_empty());
assert_eq!(log.len(), 1);
}
#[test]
fn test_activity_log_recent() {
let log = ActivityLog::new(10);
for i in 0..5 {
log.push(ActivityEntry::new(
ActionType::Pull,
format!("image:{}", i),
"docker",
"LOCAL",
));
}
let recent = log.recent(3);
assert_eq!(recent.len(), 3);
// newest first
assert_eq!(recent[0].artifact, "image:4");
assert_eq!(recent[1].artifact, "image:3");
assert_eq!(recent[2].artifact, "image:2");
}
#[test]
fn test_activity_log_all() {
let log = ActivityLog::new(10);
for i in 0..3 {
log.push(ActivityEntry::new(
ActionType::Pull,
format!("pkg:{}", i),
"npm",
"PROXY",
));
}
let all = log.all();
assert_eq!(all.len(), 3);
assert_eq!(all[0].artifact, "pkg:2"); // newest first
}
#[test]
fn test_activity_log_bounded_size() {
let log = ActivityLog::new(3);
for i in 0..5 {
log.push(ActivityEntry::new(
ActionType::Pull,
format!("item:{}", i),
"cargo",
"CACHE",
));
}
assert_eq!(log.len(), 3);
let all = log.all();
// oldest entries should be dropped
assert_eq!(all[0].artifact, "item:4");
assert_eq!(all[1].artifact, "item:3");
assert_eq!(all[2].artifact, "item:2");
}
#[test]
fn test_activity_log_recent_more_than_available() {
let log = ActivityLog::new(10);
log.push(ActivityEntry::new(
ActionType::Push,
"one".to_string(),
"maven",
"LOCAL",
));
let recent = log.recent(100);
assert_eq!(recent.len(), 1);
}
#[test]
fn test_activity_log_default() {
let log = ActivityLog::default();
assert!(log.is_empty());
// default capacity is 50
for i in 0..60 {
log.push(ActivityEntry::new(
ActionType::Pull,
format!("x:{}", i),
"docker",
"LOCAL",
));
}
assert_eq!(log.len(), 50);
}
}

View File

@@ -12,7 +12,6 @@ use serde::Serialize;
use std::fs::{self, OpenOptions}; use std::fs::{self, OpenOptions};
use std::io::Write; use std::io::Write;
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::Arc;
use tracing::{info, warn}; use tracing::{info, warn};
#[derive(Debug, Clone, Serialize)] #[derive(Debug, Clone, Serialize)]
@@ -40,7 +39,7 @@ impl AuditEntry {
pub struct AuditLog { pub struct AuditLog {
path: PathBuf, path: PathBuf,
writer: Arc<Mutex<Option<fs::File>>>, writer: Mutex<Option<fs::File>>,
} }
impl AuditLog { impl AuditLog {
@@ -49,116 +48,26 @@ impl AuditLog {
let writer = match OpenOptions::new().create(true).append(true).open(&path) { let writer = match OpenOptions::new().create(true).append(true).open(&path) {
Ok(f) => { Ok(f) => {
info!(path = %path.display(), "Audit log initialized"); info!(path = %path.display(), "Audit log initialized");
Arc::new(Mutex::new(Some(f))) Mutex::new(Some(f))
} }
Err(e) => { Err(e) => {
warn!(path = %path.display(), error = %e, "Failed to open audit log, auditing disabled"); warn!(path = %path.display(), error = %e, "Failed to open audit log, auditing disabled");
Arc::new(Mutex::new(None)) Mutex::new(None)
} }
}; };
Self { path, writer } Self { path, writer }
} }
pub fn log(&self, entry: AuditEntry) { pub fn log(&self, entry: AuditEntry) {
let writer = Arc::clone(&self.writer); if let Some(ref mut file) = *self.writer.lock() {
tokio::task::spawn_blocking(move || { if let Ok(json) = serde_json::to_string(&entry) {
if let Some(ref mut file) = *writer.lock() { let _ = writeln!(file, "{}", json);
if let Ok(json) = serde_json::to_string(&entry) { let _ = file.flush();
let _ = writeln!(file, "{}", json);
let _ = file.flush();
}
} }
}); }
} }
pub fn path(&self) -> &PathBuf { pub fn path(&self) -> &PathBuf {
&self.path &self.path
} }
} }
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
use tempfile::TempDir;
#[test]
fn test_audit_entry_new() {
let entry = AuditEntry::new(
"push",
"admin",
"nginx:latest",
"docker",
"uploaded manifest",
);
assert_eq!(entry.action, "push");
assert_eq!(entry.actor, "admin");
assert_eq!(entry.artifact, "nginx:latest");
assert_eq!(entry.registry, "docker");
assert_eq!(entry.detail, "uploaded manifest");
}
#[test]
fn test_audit_log_new_and_path() {
let tmp = TempDir::new().unwrap();
let log = AuditLog::new(tmp.path().to_str().unwrap());
assert!(log.path().ends_with("audit.jsonl"));
}
#[tokio::test]
async fn test_audit_log_write_entry() {
let tmp = TempDir::new().unwrap();
let log = AuditLog::new(tmp.path().to_str().unwrap());
let entry = AuditEntry::new("pull", "user1", "lodash", "npm", "downloaded");
log.log(entry);
// spawn_blocking is fire-and-forget; retry until flushed (max 1s)
let path = log.path().clone();
let mut content = String::new();
for _ in 0..20 {
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
content = std::fs::read_to_string(&path).unwrap_or_default();
if content.contains(r#""action":"pull""#) {
break;
}
}
assert!(content.contains(r#""action":"pull""#));
assert!(content.contains(r#""actor":"user1""#));
assert!(content.contains(r#""artifact":"lodash""#));
}
#[tokio::test]
async fn test_audit_log_multiple_entries() {
let tmp = TempDir::new().unwrap();
let log = AuditLog::new(tmp.path().to_str().unwrap());
log.log(AuditEntry::new("push", "admin", "a", "docker", ""));
log.log(AuditEntry::new("pull", "user", "b", "npm", ""));
log.log(AuditEntry::new("delete", "admin", "c", "maven", ""));
// Retry until all 3 entries flushed (max 1s)
let path = log.path().clone();
let mut line_count = 0;
for _ in 0..20 {
tokio::time::sleep(std::time::Duration::from_millis(50)).await;
if let Ok(content) = std::fs::read_to_string(&path) {
line_count = content.lines().count();
if line_count >= 3 {
break;
}
}
}
assert_eq!(line_count, 3);
}
#[test]
fn test_audit_entry_serialization() {
let entry = AuditEntry::new("push", "ci", "app:v1", "docker", "ci build");
let json = serde_json::to_string(&entry).unwrap();
assert!(json.contains(r#""action":"push""#));
assert!(json.contains(r#""ts":""#));
}
}

View File

@@ -94,16 +94,6 @@ pub async fn auth_middleware(
return next.run(request).await; return next.run(request).await;
} }
// Allow anonymous read if configured
let is_read_method = matches!(
*request.method(),
axum::http::Method::GET | axum::http::Method::HEAD
);
if state.config.auth.anonymous_read && is_read_method {
// Read requests allowed without auth
return next.run(request).await;
}
// Extract Authorization header // Extract Authorization header
let auth_header = request let auth_header = request
.headers() .headers()
@@ -366,7 +356,6 @@ pub fn token_routes() -> Router<Arc<AppState>> {
} }
#[cfg(test)] #[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests { mod tests {
use super::*; use super::*;
use std::io::Write; use std::io::Write;
@@ -404,7 +393,7 @@ mod tests {
fn test_htpasswd_loading_with_comments() { fn test_htpasswd_loading_with_comments() {
let mut file = NamedTempFile::new().unwrap(); let mut file = NamedTempFile::new().unwrap();
writeln!(file, "# This is a comment").unwrap(); writeln!(file, "# This is a comment").unwrap();
writeln!(file).unwrap(); writeln!(file, "").unwrap();
let hash = bcrypt::hash("secret", 4).unwrap(); let hash = bcrypt::hash("secret", 4).unwrap();
writeln!(file, "admin:{}", hash).unwrap(); writeln!(file, "admin:{}", hash).unwrap();
file.flush().unwrap(); file.flush().unwrap();
@@ -473,185 +462,4 @@ mod tests {
assert!(hash.starts_with("$2")); assert!(hash.starts_with("$2"));
assert!(bcrypt::verify("test123", &hash).unwrap()); assert!(bcrypt::verify("test123", &hash).unwrap());
} }
#[test]
fn test_is_public_path_health() {
assert!(is_public_path("/health"));
assert!(is_public_path("/ready"));
assert!(is_public_path("/metrics"));
}
#[test]
fn test_is_public_path_v2() {
assert!(is_public_path("/v2/"));
assert!(is_public_path("/v2"));
}
#[test]
fn test_is_public_path_ui() {
assert!(is_public_path("/ui"));
assert!(is_public_path("/ui/dashboard"));
assert!(is_public_path("/ui/repos"));
}
#[test]
fn test_is_public_path_api_docs() {
assert!(is_public_path("/api-docs"));
assert!(is_public_path("/api-docs/openapi.json"));
assert!(is_public_path("/api/ui"));
}
#[test]
fn test_is_public_path_tokens() {
assert!(is_public_path("/api/tokens"));
assert!(is_public_path("/api/tokens/list"));
assert!(is_public_path("/api/tokens/revoke"));
}
#[test]
fn test_is_public_path_root() {
assert!(is_public_path("/"));
}
#[test]
fn test_is_not_public_path_registry() {
assert!(!is_public_path("/v2/library/alpine/manifests/latest"));
assert!(!is_public_path("/npm/lodash"));
assert!(!is_public_path("/maven/com/example"));
assert!(!is_public_path("/pypi/simple/flask"));
}
#[test]
fn test_is_not_public_path_random() {
assert!(!is_public_path("/admin"));
assert!(!is_public_path("/secret"));
assert!(!is_public_path("/api/data"));
}
#[test]
fn test_default_role_str() {
assert_eq!(default_role_str(), "read");
}
#[test]
fn test_default_ttl() {
assert_eq!(default_ttl(), 30);
}
#[test]
fn test_create_token_request_defaults() {
let json = r#"{"username":"admin","password":"pass"}"#;
let req: CreateTokenRequest = serde_json::from_str(json).unwrap();
assert_eq!(req.username, "admin");
assert_eq!(req.password, "pass");
assert_eq!(req.ttl_days, 30);
assert_eq!(req.role, "read");
assert!(req.description.is_none());
}
#[test]
fn test_create_token_request_custom() {
let json = r#"{"username":"admin","password":"pass","ttl_days":90,"role":"write","description":"CI token"}"#;
let req: CreateTokenRequest = serde_json::from_str(json).unwrap();
assert_eq!(req.ttl_days, 90);
assert_eq!(req.role, "write");
assert_eq!(req.description, Some("CI token".to_string()));
}
#[test]
fn test_create_token_response_serialization() {
let resp = CreateTokenResponse {
token: "nora_abc123".to_string(),
expires_in_days: 30,
};
let json = serde_json::to_string(&resp).unwrap();
assert!(json.contains("nora_abc123"));
assert!(json.contains("30"));
}
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod integration_tests {
use crate::test_helpers::*;
use axum::http::{Method, StatusCode};
use base64::{engine::general_purpose::STANDARD, Engine};
#[tokio::test]
async fn test_auth_disabled_passes_all() {
let ctx = create_test_context();
let response = send(&ctx.app, Method::PUT, "/raw/test.txt", b"data".to_vec()).await;
assert_eq!(response.status(), StatusCode::CREATED);
}
#[tokio::test]
async fn test_auth_public_paths_always_pass() {
let ctx = create_test_context_with_auth(&[("admin", "secret")]);
let response = send(&ctx.app, Method::GET, "/health", "").await;
assert_eq!(response.status(), StatusCode::OK);
let response = send(&ctx.app, Method::GET, "/ready", "").await;
assert_eq!(response.status(), StatusCode::OK);
let response = send(&ctx.app, Method::GET, "/v2/", "").await;
assert_eq!(response.status(), StatusCode::OK);
}
#[tokio::test]
async fn test_auth_blocks_without_credentials() {
let ctx = create_test_context_with_auth(&[("admin", "secret")]);
let response = send(&ctx.app, Method::PUT, "/raw/test.txt", b"data".to_vec()).await;
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
assert!(response.headers().contains_key("www-authenticate"));
}
#[tokio::test]
async fn test_auth_basic_works() {
let ctx = create_test_context_with_auth(&[("admin", "secret")]);
let header_val = format!("Basic {}", STANDARD.encode("admin:secret"));
let response = send_with_headers(
&ctx.app,
Method::PUT,
"/raw/test.txt",
vec![("authorization", &header_val)],
b"data".to_vec(),
)
.await;
assert_eq!(response.status(), StatusCode::CREATED);
}
#[tokio::test]
async fn test_auth_basic_wrong_password() {
let ctx = create_test_context_with_auth(&[("admin", "secret")]);
let header_val = format!("Basic {}", STANDARD.encode("admin:wrong"));
let response = send_with_headers(
&ctx.app,
Method::PUT,
"/raw/test.txt",
vec![("authorization", &header_val)],
b"data".to_vec(),
)
.await;
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn test_auth_anonymous_read() {
let ctx = create_test_context_with_anonymous_read(&[("admin", "secret")]);
// Upload with auth
let header_val = format!("Basic {}", STANDARD.encode("admin:secret"));
let response = send_with_headers(
&ctx.app,
Method::PUT,
"/raw/test.txt",
vec![("authorization", &header_val)],
b"data".to_vec(),
)
.await;
assert_eq!(response.status(), StatusCode::CREATED);
// Read without auth should work
let response = send(&ctx.app, Method::GET, "/raw/test.txt", "").await;
assert_eq!(response.status(), StatusCode::OK);
// Write without auth should fail
let response = send(&ctx.app, Method::PUT, "/raw/test2.txt", b"data".to_vec()).await;
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
}
} }

View File

@@ -300,134 +300,3 @@ fn format_bytes(bytes: u64) -> String {
format!("{} B", bytes) format!("{} B", bytes)
} }
} }
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
#[test]
fn test_format_bytes_zero() {
assert_eq!(format_bytes(0), "0 B");
}
#[test]
fn test_format_bytes_bytes() {
assert_eq!(format_bytes(512), "512 B");
assert_eq!(format_bytes(1023), "1023 B");
}
#[test]
fn test_format_bytes_kilobytes() {
assert_eq!(format_bytes(1024), "1.00 KB");
assert_eq!(format_bytes(1536), "1.50 KB");
assert_eq!(format_bytes(10240), "10.00 KB");
}
#[test]
fn test_format_bytes_megabytes() {
assert_eq!(format_bytes(1048576), "1.00 MB");
assert_eq!(format_bytes(5 * 1024 * 1024), "5.00 MB");
}
#[test]
fn test_format_bytes_gigabytes() {
assert_eq!(format_bytes(1073741824), "1.00 GB");
assert_eq!(format_bytes(3 * 1024 * 1024 * 1024), "3.00 GB");
}
#[test]
fn test_backup_metadata_serialization() {
let meta = BackupMetadata {
version: "0.3.0".to_string(),
created_at: chrono::Utc::now(),
artifact_count: 42,
total_bytes: 1024000,
storage_backend: "local".to_string(),
};
let json = serde_json::to_string(&meta).unwrap();
assert!(json.contains("\"version\":\"0.3.0\""));
assert!(json.contains("\"artifact_count\":42"));
assert!(json.contains("\"storage_backend\":\"local\""));
}
#[test]
fn test_backup_metadata_deserialization() {
let json = r#"{
"version": "0.3.0",
"created_at": "2026-01-01T00:00:00Z",
"artifact_count": 10,
"total_bytes": 5000,
"storage_backend": "s3"
}"#;
let meta: BackupMetadata = serde_json::from_str(json).unwrap();
assert_eq!(meta.version, "0.3.0");
assert_eq!(meta.artifact_count, 10);
assert_eq!(meta.total_bytes, 5000);
assert_eq!(meta.storage_backend, "s3");
}
#[test]
fn test_backup_metadata_roundtrip() {
let meta = BackupMetadata {
version: "1.0.0".to_string(),
created_at: chrono::Utc::now(),
artifact_count: 100,
total_bytes: 999999,
storage_backend: "local".to_string(),
};
let json = serde_json::to_value(&meta).unwrap();
let restored: BackupMetadata = serde_json::from_value(json).unwrap();
assert_eq!(meta.version, restored.version);
assert_eq!(meta.artifact_count, restored.artifact_count);
assert_eq!(meta.total_bytes, restored.total_bytes);
}
#[tokio::test]
async fn test_create_backup_empty_storage() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
let output = dir.path().join("backup.tar.gz");
let stats = create_backup(&storage, &output).await.unwrap();
assert_eq!(stats.artifact_count, 0);
assert_eq!(stats.total_bytes, 0);
assert!(output.exists());
assert!(stats.output_size > 0); // at least metadata.json
}
#[tokio::test]
async fn test_backup_restore_roundtrip() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// Put some test data
storage
.put("maven/com/example/1.0/test.jar", b"test-content")
.await
.unwrap();
storage
.put("docker/test/blobs/sha256:abc123", b"blob-data")
.await
.unwrap();
// Create backup
let backup_file = dir.path().join("backup.tar.gz");
let backup_stats = create_backup(&storage, &backup_file).await.unwrap();
assert_eq!(backup_stats.artifact_count, 2);
// Restore to different storage
let restore_storage = Storage::new_local(dir.path().join("restored").to_str().unwrap());
let restore_stats = restore_backup(&restore_storage, &backup_file)
.await
.unwrap();
assert_eq!(restore_stats.artifact_count, 2);
// Verify data
let data = restore_storage
.get("maven/com/example/1.0/test.jar")
.await
.unwrap();
assert_eq!(&data[..], b"test-content");
}
}

View File

@@ -26,10 +26,6 @@ pub struct Config {
#[serde(default)] #[serde(default)]
pub docker: DockerConfig, pub docker: DockerConfig,
#[serde(default)] #[serde(default)]
pub go: GoConfig,
#[serde(default)]
pub cargo: CargoConfig,
#[serde(default)]
pub raw: RawConfig, pub raw: RawConfig,
#[serde(default)] #[serde(default)]
pub auth: AuthConfig, pub auth: AuthConfig,
@@ -74,10 +70,10 @@ pub struct StorageConfig {
#[serde(default = "default_bucket")] #[serde(default = "default_bucket")]
pub bucket: String, pub bucket: String,
/// S3 access key (optional, uses anonymous access if not set) /// S3 access key (optional, uses anonymous access if not set)
#[serde(default, skip_serializing)] #[serde(default)]
pub s3_access_key: Option<String>, pub s3_access_key: Option<String>,
/// S3 secret key (optional, uses anonymous access if not set) /// S3 secret key (optional, uses anonymous access if not set)
#[serde(default, skip_serializing)] #[serde(default)]
pub s3_secret_key: Option<String>, pub s3_secret_key: Option<String>,
/// S3 region (default: us-east-1) /// S3 region (default: us-east-1)
#[serde(default = "default_s3_region")] #[serde(default = "default_s3_region")]
@@ -131,74 +127,6 @@ pub struct PypiConfig {
pub proxy_timeout: u64, pub proxy_timeout: u64,
} }
/// Cargo registry configuration
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CargoConfig {
/// Upstream Cargo registry (crates.io API)
#[serde(default = "default_cargo_proxy")]
pub proxy: Option<String>,
#[serde(default)]
pub proxy_auth: Option<String>,
#[serde(default = "default_timeout")]
pub proxy_timeout: u64,
}
fn default_cargo_proxy() -> Option<String> {
Some("https://crates.io".to_string())
}
impl Default for CargoConfig {
fn default() -> Self {
Self {
proxy: default_cargo_proxy(),
proxy_auth: None,
proxy_timeout: 30,
}
}
}
/// Go module proxy configuration (GOPROXY protocol)
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GoConfig {
/// Upstream Go module proxy URL (default: https://proxy.golang.org)
#[serde(default = "default_go_proxy")]
pub proxy: Option<String>,
#[serde(default)]
pub proxy_auth: Option<String>, // "user:pass" for basic auth
#[serde(default = "default_timeout")]
pub proxy_timeout: u64,
/// Separate timeout for .zip downloads (default: 120s, zips can be large)
#[serde(default = "default_go_zip_timeout")]
pub proxy_timeout_zip: u64,
/// Maximum module zip size in bytes (default: 100MB)
#[serde(default = "default_go_max_zip_size")]
pub max_zip_size: u64,
}
fn default_go_proxy() -> Option<String> {
Some("https://proxy.golang.org".to_string())
}
fn default_go_zip_timeout() -> u64 {
120
}
fn default_go_max_zip_size() -> u64 {
104_857_600 // 100MB
}
impl Default for GoConfig {
fn default() -> Self {
Self {
proxy: default_go_proxy(),
proxy_auth: None,
proxy_timeout: 30,
proxy_timeout_zip: 120,
max_zip_size: 104_857_600,
}
}
}
/// Docker registry configuration with upstream proxy support /// Docker registry configuration with upstream proxy support
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DockerConfig { pub struct DockerConfig {
@@ -272,9 +200,6 @@ fn default_max_file_size() -> u64 {
pub struct AuthConfig { pub struct AuthConfig {
#[serde(default)] #[serde(default)]
pub enabled: bool, pub enabled: bool,
/// Allow anonymous read access (pull/download without auth, push requires auth)
#[serde(default)]
pub anonymous_read: bool,
#[serde(default = "default_htpasswd_file")] #[serde(default = "default_htpasswd_file")]
pub htpasswd_file: String, pub htpasswd_file: String,
#[serde(default = "default_token_storage")] #[serde(default = "default_token_storage")]
@@ -354,7 +279,6 @@ impl Default for AuthConfig {
fn default() -> Self { fn default() -> Self {
Self { Self {
enabled: false, enabled: false,
anonymous_read: false,
htpasswd_file: "users.htpasswd".to_string(), htpasswd_file: "users.htpasswd".to_string(),
token_storage: "data/tokens".to_string(), token_storage: "data/tokens".to_string(),
} }
@@ -442,14 +366,11 @@ impl Config {
pub fn warn_plaintext_credentials(&self) { pub fn warn_plaintext_credentials(&self) {
// Docker upstreams // Docker upstreams
for (i, upstream) in self.docker.upstreams.iter().enumerate() { for (i, upstream) in self.docker.upstreams.iter().enumerate() {
if upstream.auth.is_some() if upstream.auth.is_some() && std::env::var("NORA_DOCKER_UPSTREAMS").is_err() {
&& std::env::var("NORA_DOCKER_PROXIES").is_err()
&& std::env::var("NORA_DOCKER_UPSTREAMS").is_err()
{
tracing::warn!( tracing::warn!(
upstream_index = i, upstream_index = i,
url = %upstream.url, url = %upstream.url,
"Docker upstream credentials in config.toml are plaintext — consider NORA_DOCKER_PROXIES env var" "Docker upstream credentials in config.toml are plaintext — consider NORA_DOCKER_UPSTREAMS env var"
); );
} }
} }
@@ -462,10 +383,6 @@ impl Config {
); );
} }
} }
// Go
if self.go.proxy_auth.is_some() && std::env::var("NORA_GO_PROXY_AUTH").is_err() {
tracing::warn!("Go proxy credentials in config.toml are plaintext — consider NORA_GO_PROXY_AUTH env var");
}
// npm // npm
if self.npm.proxy_auth.is_some() && std::env::var("NORA_NPM_PROXY_AUTH").is_err() { if self.npm.proxy_auth.is_some() && std::env::var("NORA_NPM_PROXY_AUTH").is_err() {
tracing::warn!("npm proxy credentials in config.toml are plaintext — consider NORA_NPM_PROXY_AUTH env var"); tracing::warn!("npm proxy credentials in config.toml are plaintext — consider NORA_NPM_PROXY_AUTH env var");
@@ -474,72 +391,6 @@ impl Config {
if self.pypi.proxy_auth.is_some() && std::env::var("NORA_PYPI_PROXY_AUTH").is_err() { if self.pypi.proxy_auth.is_some() && std::env::var("NORA_PYPI_PROXY_AUTH").is_err() {
tracing::warn!("PyPI proxy credentials in config.toml are plaintext — consider NORA_PYPI_PROXY_AUTH env var"); tracing::warn!("PyPI proxy credentials in config.toml are plaintext — consider NORA_PYPI_PROXY_AUTH env var");
} }
// Cargo
if self.cargo.proxy_auth.is_some() && std::env::var("NORA_CARGO_PROXY_AUTH").is_err() {
tracing::warn!("Cargo proxy credentials in config.toml are plaintext — consider NORA_CARGO_PROXY_AUTH env var");
}
}
/// Validate configuration and return (warnings, errors).
///
/// Warnings are logged but do not prevent startup.
/// Errors indicate a fatal misconfiguration and should cause a panic.
pub fn validate(&self) -> (Vec<String>, Vec<String>) {
let mut warnings = Vec::new();
let mut errors = Vec::new();
// 1. Port must not be 0
if self.server.port == 0 {
errors.push("server.port must not be 0".to_string());
}
// 2. Storage path must not be empty when mode = Local
if self.storage.mode == StorageMode::Local && self.storage.path.trim().is_empty() {
errors.push("storage.path must not be empty when storage mode is local".to_string());
}
// 3. S3 bucket must not be empty when mode = S3
if self.storage.mode == StorageMode::S3 && self.storage.bucket.trim().is_empty() {
errors.push("storage.bucket must not be empty when storage mode is s3".to_string());
}
// 4. Rate limit values must be > 0 when rate limiting is enabled
if self.rate_limit.enabled {
if self.rate_limit.auth_rps == 0 {
warnings
.push("rate_limit.auth_rps is 0 while rate limiting is enabled".to_string());
}
if self.rate_limit.auth_burst == 0 {
warnings
.push("rate_limit.auth_burst is 0 while rate limiting is enabled".to_string());
}
if self.rate_limit.upload_rps == 0 {
warnings
.push("rate_limit.upload_rps is 0 while rate limiting is enabled".to_string());
}
if self.rate_limit.upload_burst == 0 {
warnings.push(
"rate_limit.upload_burst is 0 while rate limiting is enabled".to_string(),
);
}
if self.rate_limit.general_rps == 0 {
warnings
.push("rate_limit.general_rps is 0 while rate limiting is enabled".to_string());
}
if self.rate_limit.general_burst == 0 {
warnings.push(
"rate_limit.general_burst is 0 while rate limiting is enabled".to_string(),
);
}
}
// 5. Body limit must be > 0
if self.server.body_limit_mb == 0 {
warnings
.push("server.body_limit_mb is 0, no request bodies will be accepted".to_string());
}
(warnings, errors)
} }
/// Load configuration with priority: ENV > config.toml > defaults /// Load configuration with priority: ENV > config.toml > defaults
@@ -553,19 +404,6 @@ impl Config {
// 3. Override with ENV vars (highest priority) // 3. Override with ENV vars (highest priority)
config.apply_env_overrides(); config.apply_env_overrides();
// 4. Validate configuration
let (warnings, errors) = config.validate();
for w in &warnings {
tracing::warn!("Config validation: {}", w);
}
if !errors.is_empty() {
for e in &errors {
tracing::error!("Config validation: {}", e);
}
panic!("Fatal configuration errors: {}", errors.join("; "));
}
config config
} }
@@ -619,9 +457,6 @@ impl Config {
if let Ok(val) = env::var("NORA_AUTH_ENABLED") { if let Ok(val) = env::var("NORA_AUTH_ENABLED") {
self.auth.enabled = val.to_lowercase() == "true" || val == "1"; self.auth.enabled = val.to_lowercase() == "true" || val == "1";
} }
if let Ok(val) = env::var("NORA_AUTH_ANONYMOUS_READ") {
self.auth.anonymous_read = val.to_lowercase() == "true" || val == "1";
}
if let Ok(val) = env::var("NORA_AUTH_HTPASSWD_FILE") { if let Ok(val) = env::var("NORA_AUTH_HTPASSWD_FILE") {
self.auth.htpasswd_file = val; self.auth.htpasswd_file = val;
} }
@@ -691,14 +526,8 @@ impl Config {
self.docker.proxy_timeout = timeout; self.docker.proxy_timeout = timeout;
} }
} }
// NORA_DOCKER_PROXIES format: "url1,url2" or "url1|auth1,url2|auth2" // NORA_DOCKER_UPSTREAMS format: "url1,url2" or "url1|auth1,url2|auth2"
// Backward compat: NORA_DOCKER_UPSTREAMS still works but is deprecated if let Ok(val) = env::var("NORA_DOCKER_UPSTREAMS") {
if let Ok(val) =
env::var("NORA_DOCKER_PROXIES").or_else(|_| env::var("NORA_DOCKER_UPSTREAMS"))
{
if env::var("NORA_DOCKER_PROXIES").is_err() {
tracing::warn!("NORA_DOCKER_UPSTREAMS is deprecated, use NORA_DOCKER_PROXIES");
}
self.docker.upstreams = val self.docker.upstreams = val
.split(',') .split(',')
.filter(|s| !s.is_empty()) .filter(|s| !s.is_empty())
@@ -712,42 +541,6 @@ impl Config {
.collect(); .collect();
} }
// Go config
if let Ok(val) = env::var("NORA_GO_PROXY") {
self.go.proxy = if val.is_empty() { None } else { Some(val) };
}
if let Ok(val) = env::var("NORA_GO_PROXY_AUTH") {
self.go.proxy_auth = if val.is_empty() { None } else { Some(val) };
}
if let Ok(val) = env::var("NORA_GO_PROXY_TIMEOUT") {
if let Ok(timeout) = val.parse() {
self.go.proxy_timeout = timeout;
}
}
if let Ok(val) = env::var("NORA_GO_PROXY_TIMEOUT_ZIP") {
if let Ok(timeout) = val.parse() {
self.go.proxy_timeout_zip = timeout;
}
}
if let Ok(val) = env::var("NORA_GO_MAX_ZIP_SIZE") {
if let Ok(size) = val.parse() {
self.go.max_zip_size = size;
}
}
// Cargo config
if let Ok(val) = env::var("NORA_CARGO_PROXY") {
self.cargo.proxy = if val.is_empty() { None } else { Some(val) };
}
if let Ok(val) = env::var("NORA_CARGO_PROXY_TIMEOUT") {
if let Ok(timeout) = val.parse() {
self.cargo.proxy_timeout = timeout;
}
}
if let Ok(val) = env::var("NORA_CARGO_PROXY_AUTH") {
self.cargo.proxy_auth = if val.is_empty() { None } else { Some(val) };
}
// Raw config // Raw config
if let Ok(val) = env::var("NORA_RAW_ENABLED") { if let Ok(val) = env::var("NORA_RAW_ENABLED") {
self.raw.enabled = val.to_lowercase() == "true" || val == "1"; self.raw.enabled = val.to_lowercase() == "true" || val == "1";
@@ -829,8 +622,6 @@ impl Default for Config {
maven: MavenConfig::default(), maven: MavenConfig::default(),
npm: NpmConfig::default(), npm: NpmConfig::default(),
pypi: PypiConfig::default(), pypi: PypiConfig::default(),
go: GoConfig::default(),
cargo: CargoConfig::default(),
docker: DockerConfig::default(), docker: DockerConfig::default(),
raw: RawConfig::default(), raw: RawConfig::default(),
auth: AuthConfig::default(), auth: AuthConfig::default(),
@@ -841,7 +632,6 @@ impl Default for Config {
} }
#[cfg(test)] #[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests { mod tests {
use super::*; use super::*;
@@ -876,552 +666,4 @@ mod tests {
assert_eq!(config.rate_limit.upload_burst, 1000); assert_eq!(config.rate_limit.upload_burst, 1000);
assert_eq!(config.rate_limit.auth_burst, 5); // default assert_eq!(config.rate_limit.auth_burst, 5); // default
} }
#[test]
fn test_basic_auth_header() {
let header = basic_auth_header("user:pass");
assert_eq!(header, "Basic dXNlcjpwYXNz");
}
#[test]
fn test_basic_auth_header_empty() {
let header = basic_auth_header("");
assert!(header.starts_with("Basic "));
}
#[test]
fn test_config_default() {
let config = Config::default();
assert_eq!(config.server.host, "127.0.0.1");
assert_eq!(config.server.port, 4000);
assert_eq!(config.server.body_limit_mb, 2048);
assert!(config.server.public_url.is_none());
assert_eq!(config.storage.path, "data/storage");
assert_eq!(config.storage.mode, StorageMode::Local);
assert_eq!(config.storage.bucket, "registry");
assert_eq!(config.storage.s3_region, "us-east-1");
assert!(!config.auth.enabled);
assert_eq!(config.auth.htpasswd_file, "users.htpasswd");
assert_eq!(config.auth.token_storage, "data/tokens");
}
#[test]
fn test_maven_config_default() {
let m = MavenConfig::default();
assert_eq!(m.proxy_timeout, 30);
assert_eq!(m.proxies.len(), 1);
assert_eq!(m.proxies[0].url(), "https://repo1.maven.org/maven2");
assert!(m.proxies[0].auth().is_none());
}
#[test]
fn test_npm_config_default() {
let n = NpmConfig::default();
assert_eq!(n.proxy, Some("https://registry.npmjs.org".to_string()));
assert!(n.proxy_auth.is_none());
assert_eq!(n.proxy_timeout, 30);
assert_eq!(n.metadata_ttl, 300);
}
#[test]
fn test_pypi_config_default() {
let p = PypiConfig::default();
assert_eq!(p.proxy, Some("https://pypi.org/simple/".to_string()));
assert!(p.proxy_auth.is_none());
assert_eq!(p.proxy_timeout, 30);
}
#[test]
fn test_docker_config_default() {
let d = DockerConfig::default();
assert_eq!(d.proxy_timeout, 60);
assert_eq!(d.upstreams.len(), 1);
assert_eq!(d.upstreams[0].url, "https://registry-1.docker.io");
assert!(d.upstreams[0].auth.is_none());
}
#[test]
fn test_raw_config_default() {
let r = RawConfig::default();
assert!(r.enabled);
assert_eq!(r.max_file_size, 104_857_600);
}
#[test]
fn test_auth_config_default() {
let a = AuthConfig::default();
assert!(!a.enabled);
assert!(!a.anonymous_read);
assert_eq!(a.htpasswd_file, "users.htpasswd");
assert_eq!(a.token_storage, "data/tokens");
}
#[test]
fn test_auth_anonymous_read_from_toml() {
let toml = r#"
[server]
host = "127.0.0.1"
port = 4000
[storage]
mode = "local"
[auth]
enabled = true
anonymous_read = true
"#;
let config: Config = toml::from_str(toml).unwrap();
assert!(config.auth.enabled);
assert!(config.auth.anonymous_read);
}
#[test]
fn test_env_override_anonymous_read() {
let mut config = Config::default();
std::env::set_var("NORA_AUTH_ANONYMOUS_READ", "true");
config.apply_env_overrides();
assert!(config.auth.anonymous_read);
std::env::remove_var("NORA_AUTH_ANONYMOUS_READ");
}
#[test]
fn test_maven_proxy_entry_simple() {
let entry = MavenProxyEntry::Simple("https://repo.example.com".to_string());
assert_eq!(entry.url(), "https://repo.example.com");
assert!(entry.auth().is_none());
}
#[test]
fn test_maven_proxy_entry_full() {
let entry = MavenProxyEntry::Full(MavenProxy {
url: "https://private.repo.com".to_string(),
auth: Some("user:secret".to_string()),
});
assert_eq!(entry.url(), "https://private.repo.com");
assert_eq!(entry.auth(), Some("user:secret"));
}
#[test]
fn test_maven_proxy_entry_full_no_auth() {
let entry = MavenProxyEntry::Full(MavenProxy {
url: "https://repo.com".to_string(),
auth: None,
});
assert_eq!(entry.url(), "https://repo.com");
assert!(entry.auth().is_none());
}
#[test]
fn test_storage_mode_default() {
let mode = StorageMode::default();
assert_eq!(mode, StorageMode::Local);
}
#[test]
fn test_env_override_server() {
let mut config = Config::default();
std::env::set_var("NORA_HOST", "0.0.0.0");
std::env::set_var("NORA_PORT", "8080");
std::env::set_var("NORA_PUBLIC_URL", "registry.example.com");
std::env::set_var("NORA_BODY_LIMIT_MB", "4096");
config.apply_env_overrides();
assert_eq!(config.server.host, "0.0.0.0");
assert_eq!(config.server.port, 8080);
assert_eq!(
config.server.public_url,
Some("registry.example.com".to_string())
);
assert_eq!(config.server.body_limit_mb, 4096);
std::env::remove_var("NORA_HOST");
std::env::remove_var("NORA_PORT");
std::env::remove_var("NORA_PUBLIC_URL");
std::env::remove_var("NORA_BODY_LIMIT_MB");
}
#[test]
fn test_env_override_storage() {
let mut config = Config::default();
std::env::set_var("NORA_STORAGE_MODE", "s3");
std::env::set_var("NORA_STORAGE_PATH", "/data/nora");
std::env::set_var("NORA_STORAGE_BUCKET", "my-bucket");
std::env::set_var("NORA_STORAGE_S3_REGION", "eu-west-1");
config.apply_env_overrides();
assert_eq!(config.storage.mode, StorageMode::S3);
assert_eq!(config.storage.path, "/data/nora");
assert_eq!(config.storage.bucket, "my-bucket");
assert_eq!(config.storage.s3_region, "eu-west-1");
std::env::remove_var("NORA_STORAGE_MODE");
std::env::remove_var("NORA_STORAGE_PATH");
std::env::remove_var("NORA_STORAGE_BUCKET");
std::env::remove_var("NORA_STORAGE_S3_REGION");
}
#[test]
fn test_env_override_auth() {
let mut config = Config::default();
std::env::set_var("NORA_AUTH_ENABLED", "true");
std::env::set_var("NORA_AUTH_HTPASSWD_FILE", "/etc/nora/users");
std::env::set_var("NORA_AUTH_TOKEN_STORAGE", "/data/tokens");
config.apply_env_overrides();
assert!(config.auth.enabled);
assert_eq!(config.auth.htpasswd_file, "/etc/nora/users");
assert_eq!(config.auth.token_storage, "/data/tokens");
std::env::remove_var("NORA_AUTH_ENABLED");
std::env::remove_var("NORA_AUTH_HTPASSWD_FILE");
std::env::remove_var("NORA_AUTH_TOKEN_STORAGE");
}
#[test]
fn test_env_override_maven_proxies() {
let mut config = Config::default();
std::env::set_var(
"NORA_MAVEN_PROXIES",
"https://repo1.com,https://repo2.com|user:pass",
);
config.apply_env_overrides();
assert_eq!(config.maven.proxies.len(), 2);
assert_eq!(config.maven.proxies[0].url(), "https://repo1.com");
assert!(config.maven.proxies[0].auth().is_none());
assert_eq!(config.maven.proxies[1].url(), "https://repo2.com");
assert_eq!(config.maven.proxies[1].auth(), Some("user:pass"));
std::env::remove_var("NORA_MAVEN_PROXIES");
}
#[test]
fn test_env_override_npm() {
let mut config = Config::default();
std::env::set_var("NORA_NPM_PROXY", "https://npm.company.com");
std::env::set_var("NORA_NPM_PROXY_AUTH", "user:token");
std::env::set_var("NORA_NPM_PROXY_TIMEOUT", "60");
std::env::set_var("NORA_NPM_METADATA_TTL", "600");
config.apply_env_overrides();
assert_eq!(
config.npm.proxy,
Some("https://npm.company.com".to_string())
);
assert_eq!(config.npm.proxy_auth, Some("user:token".to_string()));
assert_eq!(config.npm.proxy_timeout, 60);
assert_eq!(config.npm.metadata_ttl, 600);
std::env::remove_var("NORA_NPM_PROXY");
std::env::remove_var("NORA_NPM_PROXY_AUTH");
std::env::remove_var("NORA_NPM_PROXY_TIMEOUT");
std::env::remove_var("NORA_NPM_METADATA_TTL");
}
#[test]
fn test_env_override_raw() {
let mut config = Config::default();
std::env::set_var("NORA_RAW_ENABLED", "false");
std::env::set_var("NORA_RAW_MAX_FILE_SIZE", "524288000");
config.apply_env_overrides();
assert!(!config.raw.enabled);
assert_eq!(config.raw.max_file_size, 524288000);
std::env::remove_var("NORA_RAW_ENABLED");
std::env::remove_var("NORA_RAW_MAX_FILE_SIZE");
}
#[test]
fn test_env_override_rate_limit() {
let mut config = Config::default();
std::env::set_var("NORA_RATE_LIMIT_ENABLED", "false");
std::env::set_var("NORA_RATE_LIMIT_AUTH_RPS", "10");
std::env::set_var("NORA_RATE_LIMIT_GENERAL_BURST", "500");
config.apply_env_overrides();
assert!(!config.rate_limit.enabled);
assert_eq!(config.rate_limit.auth_rps, 10);
assert_eq!(config.rate_limit.general_burst, 500);
std::env::remove_var("NORA_RATE_LIMIT_ENABLED");
std::env::remove_var("NORA_RATE_LIMIT_AUTH_RPS");
std::env::remove_var("NORA_RATE_LIMIT_GENERAL_BURST");
}
#[test]
fn test_config_from_toml_full() {
let toml = r#"
[server]
host = "0.0.0.0"
port = 8080
public_url = "nora.example.com"
body_limit_mb = 4096
[storage]
mode = "s3"
path = "/data"
s3_url = "http://minio:9000"
bucket = "artifacts"
s3_region = "eu-central-1"
[auth]
enabled = true
htpasswd_file = "/etc/nora/users.htpasswd"
[raw]
enabled = false
max_file_size = 500000000
"#;
let config: Config = toml::from_str(toml).unwrap();
assert_eq!(config.server.host, "0.0.0.0");
assert_eq!(config.server.port, 8080);
assert_eq!(
config.server.public_url,
Some("nora.example.com".to_string())
);
assert_eq!(config.server.body_limit_mb, 4096);
assert_eq!(config.storage.mode, StorageMode::S3);
assert_eq!(config.storage.s3_url, "http://minio:9000");
assert_eq!(config.storage.bucket, "artifacts");
assert!(config.auth.enabled);
assert!(!config.raw.enabled);
assert_eq!(config.raw.max_file_size, 500000000);
}
#[test]
fn test_config_from_toml_minimal() {
let toml = r#"
[server]
host = "127.0.0.1"
port = 4000
[storage]
mode = "local"
"#;
let config: Config = toml::from_str(toml).unwrap();
// Defaults should be filled
assert_eq!(config.storage.path, "data/storage");
assert_eq!(config.maven.proxies.len(), 1);
assert_eq!(
config.npm.proxy,
Some("https://registry.npmjs.org".to_string())
);
assert_eq!(config.docker.upstreams.len(), 1);
assert!(config.raw.enabled);
assert!(!config.auth.enabled);
}
#[test]
fn test_config_toml_docker_upstreams() {
let toml = r#"
[server]
host = "127.0.0.1"
port = 4000
[storage]
mode = "local"
[docker]
proxy_timeout = 120
[[docker.upstreams]]
url = "https://mirror.gcr.io"
[[docker.upstreams]]
url = "https://private.registry.io"
auth = "user:pass"
"#;
let config: Config = toml::from_str(toml).unwrap();
assert_eq!(config.docker.proxy_timeout, 120);
assert_eq!(config.docker.upstreams.len(), 2);
assert!(config.docker.upstreams[0].auth.is_none());
assert_eq!(
config.docker.upstreams[1].auth,
Some("user:pass".to_string())
);
}
#[test]
fn test_validate_default_config_ok() {
let config = Config::default();
let (warnings, errors) = config.validate();
assert!(
errors.is_empty(),
"default config should have no errors: {:?}",
errors
);
assert!(
warnings.is_empty(),
"default config should have no warnings: {:?}",
warnings
);
}
#[test]
fn test_validate_port_zero() {
let mut config = Config::default();
config.server.port = 0;
let (_, errors) = config.validate();
assert_eq!(errors.len(), 1);
assert!(errors[0].contains("port"));
}
#[test]
fn test_validate_empty_storage_path_local() {
let mut config = Config::default();
config.storage.mode = StorageMode::Local;
config.storage.path = String::new();
let (_, errors) = config.validate();
assert_eq!(errors.len(), 1);
assert!(errors[0].contains("storage.path"));
}
#[test]
fn test_validate_whitespace_storage_path_local() {
let mut config = Config::default();
config.storage.mode = StorageMode::Local;
config.storage.path = " ".to_string();
let (_, errors) = config.validate();
assert_eq!(errors.len(), 1);
assert!(errors[0].contains("storage.path"));
}
#[test]
fn test_validate_empty_bucket_s3() {
let mut config = Config::default();
config.storage.mode = StorageMode::S3;
config.storage.bucket = String::new();
let (_, errors) = config.validate();
assert_eq!(errors.len(), 1);
assert!(errors[0].contains("storage.bucket"));
}
#[test]
fn test_validate_empty_storage_path_s3_ok() {
// Empty path is fine when mode is S3
let mut config = Config::default();
config.storage.mode = StorageMode::S3;
config.storage.path = String::new();
let (_, errors) = config.validate();
assert!(errors.is_empty());
}
#[test]
fn test_validate_rate_limit_zero_rps() {
let mut config = Config::default();
config.rate_limit.enabled = true;
config.rate_limit.auth_rps = 0;
let (warnings, errors) = config.validate();
assert!(errors.is_empty());
assert_eq!(warnings.len(), 1);
assert!(warnings[0].contains("auth_rps"));
}
#[test]
fn test_validate_rate_limit_disabled_zero_ok() {
// Zero rate limit values are fine when rate limiting is disabled
let mut config = Config::default();
config.rate_limit.enabled = false;
config.rate_limit.auth_rps = 0;
config.rate_limit.auth_burst = 0;
let (warnings, errors) = config.validate();
assert!(errors.is_empty());
assert!(warnings.is_empty());
}
#[test]
fn test_validate_rate_limit_all_zeros() {
let mut config = Config::default();
config.rate_limit.enabled = true;
config.rate_limit.auth_rps = 0;
config.rate_limit.auth_burst = 0;
config.rate_limit.upload_rps = 0;
config.rate_limit.upload_burst = 0;
config.rate_limit.general_rps = 0;
config.rate_limit.general_burst = 0;
let (warnings, errors) = config.validate();
assert!(errors.is_empty());
assert_eq!(warnings.len(), 6);
}
#[test]
fn test_validate_body_limit_zero() {
let mut config = Config::default();
config.server.body_limit_mb = 0;
let (warnings, errors) = config.validate();
assert!(errors.is_empty());
assert_eq!(warnings.len(), 1);
assert!(warnings[0].contains("body_limit_mb"));
}
#[test]
fn test_validate_multiple_errors() {
let mut config = Config::default();
config.server.port = 0;
config.storage.mode = StorageMode::Local;
config.storage.path = String::new();
let (_, errors) = config.validate();
assert_eq!(errors.len(), 2);
}
#[test]
fn test_validate_warnings_and_errors_together() {
let mut config = Config::default();
config.server.port = 0;
config.server.body_limit_mb = 0;
config.rate_limit.enabled = true;
config.rate_limit.auth_rps = 0;
let (warnings, errors) = config.validate();
assert_eq!(errors.len(), 1);
assert_eq!(warnings.len(), 2); // body_limit + auth_rps
}
#[test]
fn test_env_override_docker_proxies_and_backward_compat() {
// Test new NORA_DOCKER_PROXIES name
std::env::remove_var("NORA_DOCKER_UPSTREAMS");
std::env::set_var(
"NORA_DOCKER_PROXIES",
"https://mirror.gcr.io,https://private.io|token123",
);
let mut config = Config::default();
config.apply_env_overrides();
assert_eq!(config.docker.upstreams.len(), 2);
assert_eq!(config.docker.upstreams[0].url, "https://mirror.gcr.io");
assert!(config.docker.upstreams[0].auth.is_none());
assert_eq!(config.docker.upstreams[1].url, "https://private.io");
assert_eq!(
config.docker.upstreams[1].auth,
Some("token123".to_string())
);
std::env::remove_var("NORA_DOCKER_PROXIES");
// Test backward compat: old NORA_DOCKER_UPSTREAMS still works
std::env::remove_var("NORA_DOCKER_PROXIES");
std::env::set_var("NORA_DOCKER_UPSTREAMS", "https://legacy.io|secret");
let mut config2 = Config::default();
config2.apply_env_overrides();
assert_eq!(config2.docker.upstreams.len(), 1);
assert_eq!(config2.docker.upstreams[0].url, "https://legacy.io");
assert_eq!(config2.docker.upstreams[0].auth, Some("secret".to_string()));
std::env::remove_var("NORA_DOCKER_UPSTREAMS");
}
#[test]
fn test_env_override_go_proxy() {
let mut config = Config::default();
std::env::set_var("NORA_GO_PROXY", "https://goproxy.company.com");
config.apply_env_overrides();
assert_eq!(
config.go.proxy,
Some("https://goproxy.company.com".to_string()),
);
std::env::remove_var("NORA_GO_PROXY");
}
#[test]
fn test_env_override_go_proxy_auth() {
let mut config = Config::default();
std::env::set_var("NORA_GO_PROXY_AUTH", "user:pass");
config.apply_env_overrides();
assert_eq!(config.go.proxy_auth, Some("user:pass".to_string()));
std::env::remove_var("NORA_GO_PROXY_AUTH");
}
#[test]
fn test_cargo_config_default() {
let c = CargoConfig::default();
assert_eq!(c.proxy, Some("https://crates.io".to_string()));
assert_eq!(c.proxy_timeout, 30);
}
} }

View File

@@ -2,83 +2,52 @@
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::atomic::{AtomicU64, Ordering};
use std::time::Instant; use std::time::Instant;
use tracing::{info, warn}; use tracing::{info, warn};
/// Known registry names for per-registry metrics /// Serializable snapshot of metrics for persistence
const REGISTRIES: &[&str] = &["docker", "maven", "npm", "cargo", "pypi", "raw", "go"];
/// Serializable snapshot of metrics for persistence.
/// Uses HashMap for per-registry counters — adding a new registry only
/// requires adding its name to REGISTRIES (one line).
#[derive(Serialize, Deserialize, Default)] #[derive(Serialize, Deserialize, Default)]
struct MetricsSnapshot { struct MetricsSnapshot {
downloads: u64, downloads: u64,
uploads: u64, uploads: u64,
cache_hits: u64, cache_hits: u64,
cache_misses: u64, cache_misses: u64,
#[serde(default)] docker_downloads: u64,
registry_downloads: HashMap<String, u64>, docker_uploads: u64,
#[serde(default)] npm_downloads: u64,
registry_uploads: HashMap<String, u64>, maven_downloads: u64,
maven_uploads: u64,
cargo_downloads: u64,
pypi_downloads: u64,
raw_downloads: u64,
raw_uploads: u64,
} }
/// Thread-safe atomic counter map for per-registry metrics. /// Dashboard metrics for tracking registry activity
struct CounterMap(HashMap<String, AtomicU64>); /// Uses atomic counters for thread-safe access without locks
impl CounterMap {
fn new(keys: &[&str]) -> Self {
let mut map = HashMap::with_capacity(keys.len());
for &k in keys {
map.insert(k.to_string(), AtomicU64::new(0));
}
Self(map)
}
fn inc(&self, key: &str) {
if let Some(counter) = self.0.get(key) {
counter.fetch_add(1, Ordering::Relaxed);
}
}
fn get(&self, key: &str) -> u64 {
self.0
.get(key)
.map(|c| c.load(Ordering::Relaxed))
.unwrap_or(0)
}
fn snapshot(&self) -> HashMap<String, u64> {
self.0
.iter()
.map(|(k, v)| (k.clone(), v.load(Ordering::Relaxed)))
.collect()
}
fn load_from(&self, data: &HashMap<String, u64>) {
for (k, v) in data {
if let Some(counter) = self.0.get(k.as_str()) {
counter.store(*v, Ordering::Relaxed);
}
}
}
}
/// Dashboard metrics for tracking registry activity.
/// Global counters are separate fields; per-registry counters use CounterMap.
pub struct DashboardMetrics { pub struct DashboardMetrics {
// Global counters
pub downloads: AtomicU64, pub downloads: AtomicU64,
pub uploads: AtomicU64, pub uploads: AtomicU64,
pub cache_hits: AtomicU64, pub cache_hits: AtomicU64,
pub cache_misses: AtomicU64, pub cache_misses: AtomicU64,
registry_downloads: CounterMap, // Per-registry download counters
registry_uploads: CounterMap, pub docker_downloads: AtomicU64,
pub docker_uploads: AtomicU64,
pub npm_downloads: AtomicU64,
pub maven_downloads: AtomicU64,
pub maven_uploads: AtomicU64,
pub cargo_downloads: AtomicU64,
pub pypi_downloads: AtomicU64,
pub raw_downloads: AtomicU64,
pub raw_uploads: AtomicU64,
pub start_time: Instant, pub start_time: Instant,
/// Path to metrics.json for persistence
persist_path: Option<PathBuf>, persist_path: Option<PathBuf>,
} }
@@ -89,8 +58,15 @@ impl DashboardMetrics {
uploads: AtomicU64::new(0), uploads: AtomicU64::new(0),
cache_hits: AtomicU64::new(0), cache_hits: AtomicU64::new(0),
cache_misses: AtomicU64::new(0), cache_misses: AtomicU64::new(0),
registry_downloads: CounterMap::new(REGISTRIES), docker_downloads: AtomicU64::new(0),
registry_uploads: CounterMap::new(REGISTRIES), docker_uploads: AtomicU64::new(0),
npm_downloads: AtomicU64::new(0),
maven_downloads: AtomicU64::new(0),
maven_uploads: AtomicU64::new(0),
cargo_downloads: AtomicU64::new(0),
pypi_downloads: AtomicU64::new(0),
raw_downloads: AtomicU64::new(0),
raw_uploads: AtomicU64::new(0),
start_time: Instant::now(), start_time: Instant::now(),
persist_path: None, persist_path: None,
} }
@@ -100,7 +76,9 @@ impl DashboardMetrics {
pub fn with_persistence(storage_path: &str) -> Self { pub fn with_persistence(storage_path: &str) -> Self {
let path = Path::new(storage_path).join("metrics.json"); let path = Path::new(storage_path).join("metrics.json");
let mut metrics = Self::new(); let mut metrics = Self::new();
metrics.persist_path = Some(path.clone());
// Load existing metrics if file exists
if path.exists() { if path.exists() {
match std::fs::read_to_string(&path) { match std::fs::read_to_string(&path) {
Ok(data) => match serde_json::from_str::<MetricsSnapshot>(&data) { Ok(data) => match serde_json::from_str::<MetricsSnapshot>(&data) {
@@ -109,10 +87,15 @@ impl DashboardMetrics {
metrics.uploads = AtomicU64::new(snap.uploads); metrics.uploads = AtomicU64::new(snap.uploads);
metrics.cache_hits = AtomicU64::new(snap.cache_hits); metrics.cache_hits = AtomicU64::new(snap.cache_hits);
metrics.cache_misses = AtomicU64::new(snap.cache_misses); metrics.cache_misses = AtomicU64::new(snap.cache_misses);
metrics metrics.docker_downloads = AtomicU64::new(snap.docker_downloads);
.registry_downloads metrics.docker_uploads = AtomicU64::new(snap.docker_uploads);
.load_from(&snap.registry_downloads); metrics.npm_downloads = AtomicU64::new(snap.npm_downloads);
metrics.registry_uploads.load_from(&snap.registry_uploads); metrics.maven_downloads = AtomicU64::new(snap.maven_downloads);
metrics.maven_uploads = AtomicU64::new(snap.maven_uploads);
metrics.cargo_downloads = AtomicU64::new(snap.cargo_downloads);
metrics.pypi_downloads = AtomicU64::new(snap.pypi_downloads);
metrics.raw_downloads = AtomicU64::new(snap.raw_downloads);
metrics.raw_uploads = AtomicU64::new(snap.raw_uploads);
info!( info!(
downloads = snap.downloads, downloads = snap.downloads,
uploads = snap.uploads, uploads = snap.uploads,
@@ -125,12 +108,11 @@ impl DashboardMetrics {
} }
} }
metrics.persist_path = Some(path);
metrics metrics
} }
/// Save current metrics to disk (async to avoid blocking the runtime) /// Save current metrics to disk
pub async fn save(&self) { pub fn save(&self) {
let Some(path) = &self.persist_path else { let Some(path) = &self.persist_path else {
return; return;
}; };
@@ -139,35 +121,61 @@ impl DashboardMetrics {
uploads: self.uploads.load(Ordering::Relaxed), uploads: self.uploads.load(Ordering::Relaxed),
cache_hits: self.cache_hits.load(Ordering::Relaxed), cache_hits: self.cache_hits.load(Ordering::Relaxed),
cache_misses: self.cache_misses.load(Ordering::Relaxed), cache_misses: self.cache_misses.load(Ordering::Relaxed),
registry_downloads: self.registry_downloads.snapshot(), docker_downloads: self.docker_downloads.load(Ordering::Relaxed),
registry_uploads: self.registry_uploads.snapshot(), docker_uploads: self.docker_uploads.load(Ordering::Relaxed),
npm_downloads: self.npm_downloads.load(Ordering::Relaxed),
maven_downloads: self.maven_downloads.load(Ordering::Relaxed),
maven_uploads: self.maven_uploads.load(Ordering::Relaxed),
cargo_downloads: self.cargo_downloads.load(Ordering::Relaxed),
pypi_downloads: self.pypi_downloads.load(Ordering::Relaxed),
raw_downloads: self.raw_downloads.load(Ordering::Relaxed),
raw_uploads: self.raw_uploads.load(Ordering::Relaxed),
}; };
// Atomic write: write to tmp then rename
let tmp = path.with_extension("json.tmp"); let tmp = path.with_extension("json.tmp");
if let Ok(data) = serde_json::to_string_pretty(&snap) { if let Ok(data) = serde_json::to_string_pretty(&snap) {
if tokio::fs::write(&tmp, &data).await.is_ok() { if std::fs::write(&tmp, &data).is_ok() {
let _ = tokio::fs::rename(&tmp, path).await; let _ = std::fs::rename(&tmp, path);
} }
} }
} }
/// Record a download event for the specified registry
pub fn record_download(&self, registry: &str) { pub fn record_download(&self, registry: &str) {
self.downloads.fetch_add(1, Ordering::Relaxed); self.downloads.fetch_add(1, Ordering::Relaxed);
self.registry_downloads.inc(registry); match registry {
"docker" => self.docker_downloads.fetch_add(1, Ordering::Relaxed),
"npm" => self.npm_downloads.fetch_add(1, Ordering::Relaxed),
"maven" => self.maven_downloads.fetch_add(1, Ordering::Relaxed),
"cargo" => self.cargo_downloads.fetch_add(1, Ordering::Relaxed),
"pypi" => self.pypi_downloads.fetch_add(1, Ordering::Relaxed),
"raw" => self.raw_downloads.fetch_add(1, Ordering::Relaxed),
_ => 0,
};
} }
/// Record an upload event for the specified registry
pub fn record_upload(&self, registry: &str) { pub fn record_upload(&self, registry: &str) {
self.uploads.fetch_add(1, Ordering::Relaxed); self.uploads.fetch_add(1, Ordering::Relaxed);
self.registry_uploads.inc(registry); match registry {
"docker" => self.docker_uploads.fetch_add(1, Ordering::Relaxed),
"maven" => self.maven_uploads.fetch_add(1, Ordering::Relaxed),
"raw" => self.raw_uploads.fetch_add(1, Ordering::Relaxed),
_ => 0,
};
} }
/// Record a cache hit
pub fn record_cache_hit(&self) { pub fn record_cache_hit(&self) {
self.cache_hits.fetch_add(1, Ordering::Relaxed); self.cache_hits.fetch_add(1, Ordering::Relaxed);
} }
/// Record a cache miss
pub fn record_cache_miss(&self) { pub fn record_cache_miss(&self) {
self.cache_misses.fetch_add(1, Ordering::Relaxed); self.cache_misses.fetch_add(1, Ordering::Relaxed);
} }
/// Calculate the cache hit rate as a percentage
pub fn cache_hit_rate(&self) -> f64 { pub fn cache_hit_rate(&self) -> f64 {
let hits = self.cache_hits.load(Ordering::Relaxed); let hits = self.cache_hits.load(Ordering::Relaxed);
let misses = self.cache_misses.load(Ordering::Relaxed); let misses = self.cache_misses.load(Ordering::Relaxed);
@@ -179,12 +187,27 @@ impl DashboardMetrics {
} }
} }
/// Get download count for a specific registry
pub fn get_registry_downloads(&self, registry: &str) -> u64 { pub fn get_registry_downloads(&self, registry: &str) -> u64 {
self.registry_downloads.get(registry) match registry {
"docker" => self.docker_downloads.load(Ordering::Relaxed),
"npm" => self.npm_downloads.load(Ordering::Relaxed),
"maven" => self.maven_downloads.load(Ordering::Relaxed),
"cargo" => self.cargo_downloads.load(Ordering::Relaxed),
"pypi" => self.pypi_downloads.load(Ordering::Relaxed),
"raw" => self.raw_downloads.load(Ordering::Relaxed),
_ => 0,
}
} }
/// Get upload count for a specific registry
pub fn get_registry_uploads(&self, registry: &str) -> u64 { pub fn get_registry_uploads(&self, registry: &str) -> u64 {
self.registry_uploads.get(registry) match registry {
"docker" => self.docker_uploads.load(Ordering::Relaxed),
"maven" => self.maven_uploads.load(Ordering::Relaxed),
"raw" => self.raw_uploads.load(Ordering::Relaxed),
_ => 0,
}
} }
} }
@@ -193,154 +216,3 @@ impl Default for DashboardMetrics {
Self::new() Self::new()
} }
} }
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
use tempfile::TempDir;
#[test]
fn test_new_defaults() {
let m = DashboardMetrics::new();
assert_eq!(m.downloads.load(Ordering::Relaxed), 0);
assert_eq!(m.uploads.load(Ordering::Relaxed), 0);
assert_eq!(m.cache_hits.load(Ordering::Relaxed), 0);
assert_eq!(m.cache_misses.load(Ordering::Relaxed), 0);
}
#[test]
fn test_record_download_all_registries() {
let m = DashboardMetrics::new();
for reg in &["docker", "npm", "maven", "cargo", "pypi", "go", "raw"] {
m.record_download(reg);
}
assert_eq!(m.downloads.load(Ordering::Relaxed), 7);
assert_eq!(m.get_registry_downloads("docker"), 1);
assert_eq!(m.get_registry_downloads("npm"), 1);
assert_eq!(m.get_registry_downloads("maven"), 1);
assert_eq!(m.get_registry_downloads("cargo"), 1);
assert_eq!(m.get_registry_downloads("pypi"), 1);
assert_eq!(m.get_registry_downloads("go"), 1);
assert_eq!(m.get_registry_downloads("raw"), 1);
}
#[test]
fn test_record_download_unknown_registry() {
let m = DashboardMetrics::new();
m.record_download("unknown");
assert_eq!(m.downloads.load(Ordering::Relaxed), 1);
assert_eq!(m.get_registry_downloads("docker"), 0);
}
#[test]
fn test_record_upload_all_registries() {
let m = DashboardMetrics::new();
for reg in &["docker", "npm", "maven", "cargo", "pypi", "go", "raw"] {
m.record_upload(reg);
}
assert_eq!(m.uploads.load(Ordering::Relaxed), 7);
assert_eq!(m.get_registry_uploads("docker"), 1);
assert_eq!(m.get_registry_uploads("npm"), 1);
assert_eq!(m.get_registry_uploads("maven"), 1);
assert_eq!(m.get_registry_uploads("cargo"), 1);
assert_eq!(m.get_registry_uploads("pypi"), 1);
assert_eq!(m.get_registry_uploads("go"), 1);
assert_eq!(m.get_registry_uploads("raw"), 1);
}
#[test]
fn test_record_upload_unknown_registry() {
let m = DashboardMetrics::new();
m.record_upload("npm");
assert_eq!(m.uploads.load(Ordering::Relaxed), 1);
}
#[test]
fn test_cache_hit_rate_zero() {
let m = DashboardMetrics::new();
assert_eq!(m.cache_hit_rate(), 0.0);
}
#[test]
fn test_cache_hit_rate_all_hits() {
let m = DashboardMetrics::new();
m.record_cache_hit();
m.record_cache_hit();
assert_eq!(m.cache_hit_rate(), 100.0);
}
#[test]
fn test_cache_hit_rate_mixed() {
let m = DashboardMetrics::new();
m.record_cache_hit();
m.record_cache_miss();
assert_eq!(m.cache_hit_rate(), 50.0);
}
#[test]
fn test_get_registry_downloads() {
let m = DashboardMetrics::new();
m.record_download("docker");
m.record_download("docker");
m.record_download("npm");
assert_eq!(m.get_registry_downloads("docker"), 2);
assert_eq!(m.get_registry_downloads("npm"), 1);
assert_eq!(m.get_registry_downloads("cargo"), 0);
assert_eq!(m.get_registry_downloads("unknown"), 0);
}
#[test]
fn test_get_registry_uploads() {
let m = DashboardMetrics::new();
m.record_upload("docker");
assert_eq!(m.get_registry_uploads("docker"), 1);
assert_eq!(m.get_registry_uploads("maven"), 0);
assert_eq!(m.get_registry_uploads("unknown"), 0);
}
#[tokio::test]
async fn test_persistence_save_and_load() {
let tmp = TempDir::new().unwrap();
let path = tmp.path().to_str().unwrap();
{
let m = DashboardMetrics::with_persistence(path);
m.record_download("docker");
m.record_download("docker");
m.record_upload("maven");
m.record_cache_hit();
m.save().await;
}
{
let m = DashboardMetrics::with_persistence(path);
assert_eq!(m.downloads.load(Ordering::Relaxed), 2);
assert_eq!(m.uploads.load(Ordering::Relaxed), 1);
assert_eq!(m.get_registry_downloads("docker"), 2);
assert_eq!(m.get_registry_uploads("maven"), 1);
assert_eq!(m.cache_hits.load(Ordering::Relaxed), 1);
}
}
#[test]
fn test_persistence_missing_file() {
let tmp = TempDir::new().unwrap();
let path = tmp.path().to_str().unwrap();
let m = DashboardMetrics::with_persistence(path);
assert_eq!(m.downloads.load(Ordering::Relaxed), 0);
}
#[test]
fn test_default() {
let m = DashboardMetrics::default();
assert_eq!(m.downloads.load(Ordering::Relaxed), 0);
}
#[test]
fn test_go_registry_supported() {
let m = DashboardMetrics::new();
m.record_download("go");
assert_eq!(m.get_registry_downloads("go"), 1);
}
}

View File

@@ -51,11 +51,11 @@ struct ErrorResponse {
impl IntoResponse for AppError { impl IntoResponse for AppError {
fn into_response(self) -> Response { fn into_response(self) -> Response {
let (status, message) = match self { let (status, message) = match &self {
AppError::NotFound(msg) => (StatusCode::NOT_FOUND, msg), AppError::NotFound(msg) => (StatusCode::NOT_FOUND, msg.clone()),
AppError::BadRequest(msg) => (StatusCode::BAD_REQUEST, msg), AppError::BadRequest(msg) => (StatusCode::BAD_REQUEST, msg.clone()),
AppError::Unauthorized(msg) => (StatusCode::UNAUTHORIZED, msg), AppError::Unauthorized(msg) => (StatusCode::UNAUTHORIZED, msg.clone()),
AppError::Internal(msg) => (StatusCode::INTERNAL_SERVER_ERROR, msg), AppError::Internal(msg) => (StatusCode::INTERNAL_SERVER_ERROR, msg.clone()),
AppError::Storage(e) => match e { AppError::Storage(e) => match e {
StorageError::NotFound => (StatusCode::NOT_FOUND, "Resource not found".to_string()), StorageError::NotFound => (StatusCode::NOT_FOUND, "Resource not found".to_string()),
StorageError::Validation(v) => (StatusCode::BAD_REQUEST, v.to_string()), StorageError::Validation(v) => (StatusCode::BAD_REQUEST, v.to_string()),
@@ -124,77 +124,4 @@ mod tests {
let err = AppError::NotFound("image not found".to_string()); let err = AppError::NotFound("image not found".to_string());
assert_eq!(err.to_string(), "Not found: image not found"); assert_eq!(err.to_string(), "Not found: image not found");
} }
#[test]
fn test_error_constructors() {
let err = AppError::not_found("missing");
assert!(matches!(err, AppError::NotFound(_)));
assert_eq!(err.to_string(), "Not found: missing");
let err = AppError::bad_request("invalid input");
assert!(matches!(err, AppError::BadRequest(_)));
assert_eq!(err.to_string(), "Bad request: invalid input");
let err = AppError::unauthorized("no token");
assert!(matches!(err, AppError::Unauthorized(_)));
assert_eq!(err.to_string(), "Unauthorized: no token");
let err = AppError::internal("db crashed");
assert!(matches!(err, AppError::Internal(_)));
assert_eq!(err.to_string(), "Internal error: db crashed");
}
#[test]
fn test_error_display_storage() {
let err = AppError::Storage(StorageError::NotFound);
assert!(err.to_string().contains("Storage error"));
}
#[test]
fn test_error_display_validation() {
let err = AppError::Validation(ValidationError::PathTraversal);
assert!(err.to_string().contains("Validation error"));
}
#[test]
fn test_error_into_response_not_found() {
let err = AppError::NotFound("gone".to_string());
let response = err.into_response();
assert_eq!(response.status(), StatusCode::NOT_FOUND);
}
#[test]
fn test_error_into_response_bad_request() {
let err = AppError::BadRequest("bad".to_string());
let response = err.into_response();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
}
#[test]
fn test_error_into_response_unauthorized() {
let err = AppError::Unauthorized("nope".to_string());
let response = err.into_response();
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
}
#[test]
fn test_error_into_response_internal() {
let err = AppError::Internal("boom".to_string());
let response = err.into_response();
assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR);
}
#[test]
fn test_error_into_response_storage_not_found() {
let err = AppError::Storage(StorageError::NotFound);
let response = err.into_response();
assert_eq!(response.status(), StatusCode::NOT_FOUND);
}
#[test]
fn test_error_into_response_validation() {
let err = AppError::Validation(ValidationError::EmptyInput);
let response = err.into_response();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
}
} }

View File

@@ -1,14 +1,10 @@
//! Garbage Collection for orphaned Docker blobs //! Garbage Collection for orphaned blobs
//! //!
//! Mark-and-sweep approach: //! Mark-and-sweep approach:
//! 1. List all Docker blobs //! 1. List all blobs across registries
//! 2. Parse Docker manifests to find referenced blobs //! 2. Parse all manifests to find referenced blobs
//! 3. Blobs not referenced by any manifest = orphans //! 3. Blobs not referenced by any manifest = orphans
//! 4. Delete orphans (with --dry-run support) //! 4. Delete orphans (with --dry-run support)
//!
//! Currently Docker-only. Other registries (npm, maven, cargo, pypi, go,
//! raw) are excluded because no reference resolver exists for their
//! metadata formats.
use std::collections::HashSet; use std::collections::HashSet;
@@ -76,13 +72,8 @@ pub async fn run_gc(storage: &Storage, dry_run: bool) -> GcResult {
async fn collect_all_blobs(storage: &Storage) -> Vec<String> { async fn collect_all_blobs(storage: &Storage) -> Vec<String> {
let mut blobs = Vec::new(); let mut blobs = Vec::new();
// Only collect Docker blobs. Other registries (npm, maven, cargo, pypi, let docker_blobs = storage.list("docker/").await;
// go, raw) use storage key schemes that collect_referenced_digests does for key in docker_blobs {
// not understand, so their artifacts would appear as orphans and be
// deleted. Extending GC to non-Docker registries requires per-registry
// reference resolution.
let keys = storage.list("docker/").await;
for key in keys {
if key.contains("/blobs/") { if key.contains("/blobs/") {
blobs.push(key); blobs.push(key);
} }
@@ -128,283 +119,3 @@ async fn collect_referenced_digests(storage: &Storage) -> HashSet<String> {
referenced referenced
} }
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
#[test]
fn test_gc_result_defaults() {
let result = GcResult {
total_blobs: 0,
referenced_blobs: 0,
orphaned_blobs: 0,
deleted_blobs: 0,
orphan_keys: vec![],
};
assert_eq!(result.total_blobs, 0);
assert!(result.orphan_keys.is_empty());
}
#[tokio::test]
async fn test_gc_empty_storage() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
let result = run_gc(&storage, true).await;
assert_eq!(result.total_blobs, 0);
assert_eq!(result.referenced_blobs, 0);
assert_eq!(result.orphaned_blobs, 0);
assert_eq!(result.deleted_blobs, 0);
}
#[tokio::test]
async fn test_gc_no_orphans() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// Create a manifest that references a blob
let manifest = serde_json::json!({
"config": {"digest": "sha256:configabc"},
"layers": [{"digest": "sha256:layer111", "size": 100}]
});
storage
.put(
"docker/test/manifests/latest.json",
manifest.to_string().as_bytes(),
)
.await
.unwrap();
storage
.put("docker/test/blobs/sha256:configabc", b"config-data")
.await
.unwrap();
storage
.put("docker/test/blobs/sha256:layer111", b"layer-data")
.await
.unwrap();
let result = run_gc(&storage, true).await;
assert_eq!(result.total_blobs, 2);
assert_eq!(result.orphaned_blobs, 0);
}
#[tokio::test]
async fn test_gc_finds_orphans_dry_run() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// Create a manifest referencing only one blob
let manifest = serde_json::json!({
"config": {"digest": "sha256:configabc"},
"layers": [{"digest": "sha256:layer111", "size": 100}]
});
storage
.put(
"docker/test/manifests/latest.json",
manifest.to_string().as_bytes(),
)
.await
.unwrap();
storage
.put("docker/test/blobs/sha256:configabc", b"config-data")
.await
.unwrap();
storage
.put("docker/test/blobs/sha256:layer111", b"layer-data")
.await
.unwrap();
// Orphan blob (not referenced)
storage
.put("docker/test/blobs/sha256:orphan999", b"orphan-data")
.await
.unwrap();
let result = run_gc(&storage, true).await;
assert_eq!(result.total_blobs, 3);
assert_eq!(result.orphaned_blobs, 1);
assert_eq!(result.deleted_blobs, 0); // dry run
assert!(result.orphan_keys[0].contains("orphan999"));
// Verify orphan still exists (dry run)
assert!(storage
.get("docker/test/blobs/sha256:orphan999")
.await
.is_ok());
}
#[tokio::test]
async fn test_gc_deletes_orphans() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
let manifest = serde_json::json!({
"config": {"digest": "sha256:configabc"},
"layers": []
});
storage
.put(
"docker/test/manifests/latest.json",
manifest.to_string().as_bytes(),
)
.await
.unwrap();
storage
.put("docker/test/blobs/sha256:configabc", b"config")
.await
.unwrap();
storage
.put("docker/test/blobs/sha256:orphan1", b"orphan")
.await
.unwrap();
let result = run_gc(&storage, false).await;
assert_eq!(result.orphaned_blobs, 1);
assert_eq!(result.deleted_blobs, 1);
// Verify orphan is gone
assert!(storage
.get("docker/test/blobs/sha256:orphan1")
.await
.is_err());
// Referenced blob still exists
assert!(storage
.get("docker/test/blobs/sha256:configabc")
.await
.is_ok());
}
#[tokio::test]
async fn test_gc_manifest_list_references() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// Multi-arch manifest list
let manifest = serde_json::json!({
"manifests": [
{"digest": "sha256:platformA", "size": 100},
{"digest": "sha256:platformB", "size": 200}
]
});
storage
.put(
"docker/multi/manifests/latest.json",
manifest.to_string().as_bytes(),
)
.await
.unwrap();
storage
.put("docker/multi/blobs/sha256:platformA", b"arch-a")
.await
.unwrap();
storage
.put("docker/multi/blobs/sha256:platformB", b"arch-b")
.await
.unwrap();
let result = run_gc(&storage, true).await;
assert_eq!(result.orphaned_blobs, 0);
}
#[tokio::test]
async fn test_gc_ignores_non_docker_registries() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// Non-Docker artifacts must not be collected by GC, because
// collect_referenced_digests only understands Docker manifests.
// Without this guard, these would all appear as orphans and be deleted.
storage
.put("npm/lodash/tarballs/lodash-4.17.21.tgz", b"tarball-data")
.await
.unwrap();
storage
.put("maven/com/example/lib/1.0/lib-1.0.jar", b"jar-data")
.await
.unwrap();
storage
.put("cargo/serde/1.0.0/serde-1.0.0.crate", b"crate-data")
.await
.unwrap();
let result = run_gc(&storage, true).await;
assert_eq!(result.total_blobs, 0);
assert_eq!(result.orphaned_blobs, 0);
}
#[tokio::test]
async fn test_gc_does_not_delete_npm_tarballs() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// Regression test: npm tarballs were previously collected because
// their keys contain "/tarballs/", but no reference resolver existed
// for npm metadata, so they were all treated as orphans.
storage
.put("npm/lodash/tarballs/lodash-4.17.21.tgz", b"tarball-data")
.await
.unwrap();
let result = run_gc(&storage, false).await;
assert_eq!(result.deleted_blobs, 0);
// Verify tarball still exists
assert!(storage
.get("npm/lodash/tarballs/lodash-4.17.21.tgz")
.await
.is_ok());
}
#[tokio::test]
async fn test_gc_deletes_docker_orphan_but_preserves_npm() {
let dir = tempfile::tempdir().unwrap();
let storage = Storage::new_local(dir.path().join("data").to_str().unwrap());
// Docker manifest referencing one blob
let manifest = serde_json::json!({
"config": {"digest": "sha256:configabc"},
"layers": []
});
storage
.put(
"docker/test/manifests/latest.json",
manifest.to_string().as_bytes(),
)
.await
.unwrap();
storage
.put("docker/test/blobs/sha256:configabc", b"config")
.await
.unwrap();
// Orphan Docker blob
storage
.put("docker/test/blobs/sha256:orphan1", b"orphan")
.await
.unwrap();
// npm tarball that must survive
storage
.put("npm/lodash/tarballs/lodash-4.17.21.tgz", b"tarball-data")
.await
.unwrap();
let result = run_gc(&storage, false).await;
assert_eq!(result.total_blobs, 2); // only Docker blobs counted
assert_eq!(result.orphaned_blobs, 1);
assert_eq!(result.deleted_blobs, 1);
// Docker orphan gone
assert!(storage
.get("docker/test/blobs/sha256:orphan1")
.await
.is_err());
// Docker referenced blob still exists
assert!(storage
.get("docker/test/blobs/sha256:configabc")
.await
.is_ok());
// npm tarball untouched
assert!(storage
.get("npm/lodash/tarballs/lodash-4.17.21.tgz")
.await
.is_ok());
}
}

View File

@@ -21,7 +21,6 @@ pub struct StorageHealth {
pub backend: String, pub backend: String,
pub reachable: bool, pub reachable: bool,
pub endpoint: String, pub endpoint: String,
pub total_size_bytes: u64,
} }
#[derive(Serialize)] #[derive(Serialize)]
@@ -31,8 +30,6 @@ pub struct RegistriesHealth {
pub npm: String, pub npm: String,
pub cargo: String, pub cargo: String,
pub pypi: String, pub pypi: String,
pub go: String,
pub raw: String,
} }
pub fn routes() -> Router<Arc<AppState>> { pub fn routes() -> Router<Arc<AppState>> {
@@ -43,7 +40,6 @@ pub fn routes() -> Router<Arc<AppState>> {
async fn health_check(State(state): State<Arc<AppState>>) -> (StatusCode, Json<HealthStatus>) { async fn health_check(State(state): State<Arc<AppState>>) -> (StatusCode, Json<HealthStatus>) {
let storage_reachable = check_storage_reachable(&state).await; let storage_reachable = check_storage_reachable(&state).await;
let total_size = state.storage.total_size().await;
let status = if storage_reachable { let status = if storage_reachable {
"healthy" "healthy"
@@ -64,7 +60,6 @@ async fn health_check(State(state): State<Arc<AppState>>) -> (StatusCode, Json<H
"s3" => state.config.storage.s3_url.clone(), "s3" => state.config.storage.s3_url.clone(),
_ => state.config.storage.path.clone(), _ => state.config.storage.path.clone(),
}, },
total_size_bytes: total_size,
}, },
registries: RegistriesHealth { registries: RegistriesHealth {
docker: "ok".to_string(), docker: "ok".to_string(),
@@ -72,8 +67,6 @@ async fn health_check(State(state): State<Arc<AppState>>) -> (StatusCode, Json<H
npm: "ok".to_string(), npm: "ok".to_string(),
cargo: "ok".to_string(), cargo: "ok".to_string(),
pypi: "ok".to_string(), pypi: "ok".to_string(),
go: "ok".to_string(),
raw: "ok".to_string(),
}, },
}; };
@@ -97,72 +90,3 @@ async fn readiness_check(State(state): State<Arc<AppState>>) -> StatusCode {
async fn check_storage_reachable(state: &AppState) -> bool { async fn check_storage_reachable(state: &AppState) -> bool {
state.storage.health_check().await state.storage.health_check().await
} }
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use crate::test_helpers::{body_bytes, create_test_context, send};
use axum::http::{Method, StatusCode};
#[tokio::test]
async fn test_health_returns_200() {
let ctx = create_test_context();
let response = send(&ctx.app, Method::GET, "/health", "").await;
assert_eq!(response.status(), StatusCode::OK);
let body = body_bytes(response).await;
let body_str = std::str::from_utf8(&body).unwrap();
assert!(body_str.contains("healthy"));
}
#[tokio::test]
async fn test_health_json_has_version() {
let ctx = create_test_context();
let response = send(&ctx.app, Method::GET, "/health", "").await;
assert_eq!(response.status(), StatusCode::OK);
let body = body_bytes(response).await;
let json: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert!(json.get("version").is_some());
}
#[tokio::test]
async fn test_health_json_has_storage_size() {
let ctx = create_test_context();
// Put some data to have non-zero size
ctx.state
.storage
.put("test/artifact", b"hello world")
.await
.unwrap();
let response = send(&ctx.app, Method::GET, "/health", "").await;
assert_eq!(response.status(), StatusCode::OK);
let body = body_bytes(response).await;
let json: serde_json::Value = serde_json::from_slice(&body).unwrap();
let storage = json.get("storage").unwrap();
let size = storage.get("total_size_bytes").unwrap().as_u64().unwrap();
assert!(
size > 0,
"total_size_bytes should be > 0 after storing data"
);
}
#[tokio::test]
async fn test_health_empty_storage_size_zero() {
let ctx = create_test_context();
let response = send(&ctx.app, Method::GET, "/health", "").await;
let body = body_bytes(response).await;
let json: serde_json::Value = serde_json::from_slice(&body).unwrap();
let size = json["storage"]["total_size_bytes"].as_u64().unwrap();
assert_eq!(size, 0, "empty storage should report 0 bytes");
}
#[tokio::test]
async fn test_ready_returns_200() {
let ctx = create_test_context();
let response = send(&ctx.app, Method::GET, "/ready", "").await;
assert_eq!(response.status(), StatusCode::OK);
}
}

View File

@@ -1,30 +0,0 @@
#![deny(clippy::unwrap_used)]
#![forbid(unsafe_code)]
//! NORA Registry — library interface for fuzzing and testing
pub mod validation;
/// Re-export Docker manifest parsing for fuzz targets
pub mod docker_fuzz {
pub fn detect_manifest_media_type(data: &[u8]) -> String {
let Ok(value) = serde_json::from_slice::<serde_json::Value>(data) else {
return "application/octet-stream".to_string();
};
if let Some(mt) = value.get("mediaType").and_then(|v| v.as_str()) {
return mt.to_string();
}
if value.get("manifests").is_some() {
return "application/vnd.oci.image.index.v1+json".to_string();
}
if value.get("schemaVersion").and_then(|v| v.as_i64()) == Some(2) {
if value.get("layers").is_some() {
return "application/vnd.oci.image.manifest.v1+json".to_string();
}
return "application/vnd.docker.distribution.manifest.v2+json".to_string();
}
if value.get("schemaVersion").and_then(|v| v.as_i64()) == Some(1) {
return "application/vnd.docker.distribution.manifest.v1+json".to_string();
}
"application/vnd.docker.distribution.manifest.v2+json".to_string()
}
}

View File

@@ -1,7 +1,6 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay // Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
#![deny(clippy::unwrap_used)]
#![forbid(unsafe_code)]
mod activity_log; mod activity_log;
mod audit; mod audit;
mod auth; mod auth;
@@ -13,7 +12,6 @@ mod gc;
mod health; mod health;
mod metrics; mod metrics;
mod migrate; mod migrate;
mod mirror;
mod openapi; mod openapi;
mod rate_limit; mod rate_limit;
mod registry; mod registry;
@@ -25,10 +23,7 @@ mod tokens;
mod ui; mod ui;
mod validation; mod validation;
#[cfg(test)] use axum::{extract::DefaultBodyLimit, middleware, Router};
mod test_helpers;
use axum::{extract::DefaultBodyLimit, http::HeaderValue, middleware, Router};
use clap::{Parser, Subcommand}; use clap::{Parser, Subcommand};
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::Arc; use std::sync::Arc;
@@ -46,9 +41,6 @@ use repo_index::RepoIndex;
pub use storage::Storage; pub use storage::Storage;
use tokens::TokenStore; use tokens::TokenStore;
use parking_lot::RwLock;
use std::collections::HashMap;
#[derive(Parser)] #[derive(Parser)]
#[command(name = "nora", version, about = "Multi-protocol artifact registry")] #[command(name = "nora", version, about = "Multi-protocol artifact registry")]
struct Cli { struct Cli {
@@ -90,20 +82,6 @@ enum Commands {
#[arg(long, default_value = "false")] #[arg(long, default_value = "false")]
dry_run: bool, dry_run: bool,
}, },
/// Pre-fetch dependencies through NORA proxy cache
Mirror {
#[command(subcommand)]
format: mirror::MirrorFormat,
/// NORA registry URL
#[arg(long, default_value = "http://localhost:4000", global = true)]
registry: String,
/// Max concurrent downloads
#[arg(long, default_value = "8", global = true)]
concurrency: usize,
/// Output results as JSON (for CI pipelines)
#[arg(long, global = true)]
json: bool,
},
} }
pub struct AppState { pub struct AppState {
@@ -118,7 +96,6 @@ pub struct AppState {
pub docker_auth: registry::DockerAuth, pub docker_auth: registry::DockerAuth,
pub repo_index: RepoIndex, pub repo_index: RepoIndex,
pub http_client: reqwest::Client, pub http_client: reqwest::Client,
pub upload_sessions: Arc<RwLock<HashMap<String, registry::docker::UploadSession>>>,
} }
#[tokio::main] #[tokio::main]
@@ -187,17 +164,6 @@ async fn main() {
println!("\nRun without --dry-run to delete orphaned blobs."); println!("\nRun without --dry-run to delete orphaned blobs.");
} }
} }
Some(Commands::Mirror {
format,
registry,
concurrency,
json,
}) => {
if let Err(e) = mirror::run_mirror(format, &registry, concurrency, json).await {
error!("Mirror failed: {}", e);
std::process::exit(1);
}
}
Some(Commands::Migrate { from, to, dry_run }) => { Some(Commands::Migrate { from, to, dry_run }) => {
let source = match from.as_str() { let source = match from.as_str() {
"local" => Storage::new_local(&config.storage.path), "local" => Storage::new_local(&config.storage.path),
@@ -338,8 +304,7 @@ async fn run_server(config: Config, storage: Storage) {
.merge(registry::npm_routes()) .merge(registry::npm_routes())
.merge(registry::cargo_routes()) .merge(registry::cargo_routes())
.merge(registry::pypi_routes()) .merge(registry::pypi_routes())
.merge(registry::raw_routes()) .merge(registry::raw_routes());
.merge(registry::go_routes());
// Routes WITHOUT rate limiting (health, metrics, UI) // Routes WITHOUT rate limiting (health, metrics, UI)
let public_routes = Router::new() let public_routes = Router::new()
@@ -380,7 +345,6 @@ async fn run_server(config: Config, storage: Storage) {
docker_auth, docker_auth,
repo_index: RepoIndex::new(), repo_index: RepoIndex::new(),
http_client, http_client,
upload_sessions: Arc::new(RwLock::new(HashMap::new())),
}); });
let app = Router::new() let app = Router::new()
@@ -389,22 +353,6 @@ async fn run_server(config: Config, storage: Storage) {
.layer(DefaultBodyLimit::max( .layer(DefaultBodyLimit::max(
state.config.server.body_limit_mb * 1024 * 1024, state.config.server.body_limit_mb * 1024 * 1024,
)) ))
.layer(tower_http::set_header::SetResponseHeaderLayer::overriding(
axum::http::header::HeaderName::from_static("x-content-type-options"),
HeaderValue::from_static("nosniff"),
))
.layer(tower_http::set_header::SetResponseHeaderLayer::overriding(
axum::http::header::HeaderName::from_static("x-frame-options"),
HeaderValue::from_static("DENY"),
))
.layer(tower_http::set_header::SetResponseHeaderLayer::overriding(
axum::http::header::HeaderName::from_static("referrer-policy"),
HeaderValue::from_static("strict-origin-when-cross-origin"),
))
.layer(tower_http::set_header::SetResponseHeaderLayer::overriding(
axum::http::header::HeaderName::from_static("content-security-policy"),
HeaderValue::from_static("default-src 'self'; script-src 'self' 'unsafe-inline' https://cdn.tailwindcss.com https://unpkg.com; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self'; connect-src 'self'"),
))
.layer(middleware::from_fn(request_id::request_id_middleware)) .layer(middleware::from_fn(request_id::request_id_middleware))
.layer(middleware::from_fn(metrics::metrics_middleware)) .layer(middleware::from_fn(metrics::metrics_middleware))
.layer(middleware::from_fn_with_state( .layer(middleware::from_fn_with_state(
@@ -438,22 +386,17 @@ async fn run_server(config: Config, storage: Storage) {
npm = "/npm/", npm = "/npm/",
cargo = "/cargo/", cargo = "/cargo/",
pypi = "/simple/", pypi = "/simple/",
go = "/go/",
raw = "/raw/", raw = "/raw/",
"Available endpoints" "Available endpoints"
); );
// Background task: persist metrics and flush token last_used every 30 seconds // Background task: persist metrics every 30 seconds
let metrics_state = state.clone(); let metrics_state = state.clone();
tokio::spawn(async move { tokio::spawn(async move {
let mut interval = tokio::time::interval(std::time::Duration::from_secs(30)); let mut interval = tokio::time::interval(std::time::Duration::from_secs(30));
loop { loop {
interval.tick().await; interval.tick().await;
metrics_state.metrics.save().await; metrics_state.metrics.save();
if let Some(ref token_store) = metrics_state.tokens {
token_store.flush_last_used().await;
}
registry::docker::cleanup_expired_sessions(&metrics_state.upload_sessions);
} }
}); });
@@ -467,7 +410,7 @@ async fn run_server(config: Config, storage: Storage) {
.expect("Server error"); .expect("Server error");
// Save metrics on shutdown // Save metrics on shutdown
state.metrics.save().await; state.metrics.save();
info!( info!(
uptime_seconds = state.start_time.elapsed().as_secs(), uptime_seconds = state.start_time.elapsed().as_secs(),

View File

@@ -26,7 +26,7 @@ lazy_static! {
"nora_http_requests_total", "nora_http_requests_total",
"Total number of HTTP requests", "Total number of HTTP requests",
&["registry", "method", "status"] &["registry", "method", "status"]
).expect("failed to create HTTP_REQUESTS_TOTAL metric at startup"); ).expect("metric can be created");
/// HTTP request duration histogram /// HTTP request duration histogram
pub static ref HTTP_REQUEST_DURATION: HistogramVec = register_histogram_vec!( pub static ref HTTP_REQUEST_DURATION: HistogramVec = register_histogram_vec!(
@@ -34,28 +34,28 @@ lazy_static! {
"HTTP request latency in seconds", "HTTP request latency in seconds",
&["registry", "method"], &["registry", "method"],
vec![0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0] vec![0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0]
).expect("failed to create HTTP_REQUEST_DURATION metric at startup"); ).expect("metric can be created");
/// Cache requests counter (hit/miss) /// Cache requests counter (hit/miss)
pub static ref CACHE_REQUESTS: IntCounterVec = register_int_counter_vec!( pub static ref CACHE_REQUESTS: IntCounterVec = register_int_counter_vec!(
"nora_cache_requests_total", "nora_cache_requests_total",
"Total cache requests", "Total cache requests",
&["registry", "result"] &["registry", "result"]
).expect("failed to create CACHE_REQUESTS metric at startup"); ).expect("metric can be created");
/// Storage operations counter /// Storage operations counter
pub static ref STORAGE_OPERATIONS: IntCounterVec = register_int_counter_vec!( pub static ref STORAGE_OPERATIONS: IntCounterVec = register_int_counter_vec!(
"nora_storage_operations_total", "nora_storage_operations_total",
"Total storage operations", "Total storage operations",
&["operation", "status"] &["operation", "status"]
).expect("failed to create STORAGE_OPERATIONS metric at startup"); ).expect("metric can be created");
/// Artifacts count by registry /// Artifacts count by registry
pub static ref ARTIFACTS_TOTAL: IntCounterVec = register_int_counter_vec!( pub static ref ARTIFACTS_TOTAL: IntCounterVec = register_int_counter_vec!(
"nora_artifacts_total", "nora_artifacts_total",
"Total artifacts stored", "Total artifacts stored",
&["registry"] &["registry"]
).expect("failed to create ARTIFACTS_TOTAL metric at startup"); ).expect("metric can be created");
} }
/// Routes for metrics endpoint /// Routes for metrics endpoint
@@ -121,10 +121,6 @@ fn detect_registry(path: &str) -> String {
"cargo".to_string() "cargo".to_string()
} else if path.starts_with("/simple") || path.starts_with("/packages") { } else if path.starts_with("/simple") || path.starts_with("/packages") {
"pypi".to_string() "pypi".to_string()
} else if path.starts_with("/go/") {
"go".to_string()
} else if path.starts_with("/raw/") {
"raw".to_string()
} else if path.starts_with("/ui") { } else if path.starts_with("/ui") {
"ui".to_string() "ui".to_string()
} else { } else {
@@ -152,96 +148,3 @@ pub fn record_storage_op(operation: &str, success: bool) {
.with_label_values(&[operation, status]) .with_label_values(&[operation, status])
.inc(); .inc();
} }
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_detect_registry_docker() {
assert_eq!(detect_registry("/v2/nginx/manifests/latest"), "docker");
assert_eq!(detect_registry("/v2/"), "docker");
assert_eq!(
detect_registry("/v2/library/alpine/blobs/sha256:abc"),
"docker"
);
}
#[test]
fn test_detect_registry_maven() {
assert_eq!(detect_registry("/maven2/com/example/artifact"), "maven");
}
#[test]
fn test_detect_registry_npm() {
assert_eq!(detect_registry("/npm/lodash"), "npm");
assert_eq!(detect_registry("/npm/@scope/package"), "npm");
}
#[test]
fn test_detect_registry_cargo_path() {
assert_eq!(detect_registry("/cargo/api/v1/crates"), "cargo");
}
#[test]
fn test_detect_registry_pypi() {
assert_eq!(detect_registry("/simple/requests/"), "pypi");
assert_eq!(
detect_registry("/packages/requests/1.0/requests-1.0.tar.gz"),
"pypi"
);
}
#[test]
fn test_detect_registry_ui() {
assert_eq!(detect_registry("/ui/dashboard"), "ui");
assert_eq!(detect_registry("/ui"), "ui");
}
#[test]
fn test_detect_registry_other() {
assert_eq!(detect_registry("/health"), "other");
assert_eq!(detect_registry("/ready"), "other");
assert_eq!(detect_registry("/unknown/path"), "other");
}
#[test]
fn test_detect_registry_go_path() {
assert_eq!(
detect_registry("/go/github.com/user/repo/@v/v1.0.0.info"),
"go"
);
assert_eq!(detect_registry("/go/github.com/user/repo/@latest"), "go");
// Bare prefix without trailing slash should not match
assert_eq!(detect_registry("/goblin/something"), "other");
}
#[test]
fn test_detect_registry_raw_path() {
assert_eq!(detect_registry("/raw/my-project/artifact.tar.gz"), "raw");
assert_eq!(detect_registry("/raw/data/file.bin"), "raw");
// Bare prefix without trailing slash should not match
assert_eq!(detect_registry("/rawdata/file"), "other");
}
#[test]
fn test_record_cache_hit() {
record_cache_hit("docker");
// Doesn't panic — metric is recorded
}
#[test]
fn test_record_cache_miss() {
record_cache_miss("npm");
}
#[test]
fn test_record_storage_op_success() {
record_storage_op("get", true);
}
#[test]
fn test_record_storage_op_error() {
record_storage_op("put", false);
}
}

View File

@@ -138,7 +138,6 @@ pub async fn migrate(
} }
#[cfg(test)] #[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests { mod tests {
use super::*; use super::*;
use tempfile::TempDir; use tempfile::TempDir;
@@ -202,9 +201,16 @@ mod tests {
src.put("test/file", b"data").await.unwrap(); src.put("test/file", b"data").await.unwrap();
let stats = migrate(&src, &dst, MigrateOptions { dry_run: true }) let stats = migrate(
.await &src,
.unwrap(); &dst,
MigrateOptions {
dry_run: true,
..Default::default()
},
)
.await
.unwrap();
assert_eq!(stats.migrated, 1); assert_eq!(stats.migrated, 1);

View File

@@ -1,610 +0,0 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT
//! Docker image mirroring — fetch images from upstream registries and push to NORA.
use super::{create_progress_bar, MirrorResult};
use crate::registry::docker_auth::DockerAuth;
use reqwest::Client;
use std::time::Duration;
const DEFAULT_REGISTRY: &str = "https://registry-1.docker.io";
const DEFAULT_TIMEOUT: u64 = 120;
/// Parsed Docker image reference
#[derive(Debug, Clone, PartialEq)]
pub struct ImageRef {
/// Upstream registry (e.g., "registry-1.docker.io", "ghcr.io")
pub registry: String,
/// Image name (e.g., "library/alpine", "grafana/grafana")
pub name: String,
/// Tag or digest reference (e.g., "3.20", "sha256:abc...")
pub reference: String,
}
/// Parse an image reference string into structured components.
///
/// Supports formats:
/// - `alpine:3.20` → Docker Hub library/alpine:3.20
/// - `grafana/grafana:latest` → Docker Hub grafana/grafana:latest
/// - `ghcr.io/owner/repo:v1` → ghcr.io owner/repo:v1
/// - `alpine@sha256:abc` → Docker Hub library/alpine@sha256:abc
/// - `alpine` → Docker Hub library/alpine:latest
pub fn parse_image_ref(input: &str) -> ImageRef {
let input = input.trim();
// Split off @digest or :tag
let (name_part, reference) = if let Some(idx) = input.rfind('@') {
(&input[..idx], &input[idx + 1..])
} else if let Some(idx) = input.rfind(':') {
// Make sure colon is not part of a port (e.g., localhost:5000/image)
let before_colon = &input[..idx];
if let Some(last_slash) = before_colon.rfind('/') {
let segment_after_slash = &input[last_slash + 1..];
if segment_after_slash.contains(':') {
// Colon in last segment — tag separator
(&input[..idx], &input[idx + 1..])
} else {
// Colon in earlier segment (port) — no tag
(input, "latest")
}
} else {
(&input[..idx], &input[idx + 1..])
}
} else {
(input, "latest")
};
// Determine if first segment is a registry hostname
let parts: Vec<&str> = name_part.splitn(2, '/').collect();
let (registry, name) = if parts.len() == 1 {
// Simple name like "alpine" → Docker Hub library/
(
DEFAULT_REGISTRY.to_string(),
format!("library/{}", parts[0]),
)
} else {
let first = parts[0];
// A segment is a registry if it contains a dot or colon (hostname/port)
if first.contains('.') || first.contains(':') {
let reg = if first.starts_with("http") {
first.to_string()
} else {
format!("https://{}", first)
};
(reg, parts[1].to_string())
} else {
// Docker Hub with org, e.g., "grafana/grafana"
(DEFAULT_REGISTRY.to_string(), name_part.to_string())
}
};
ImageRef {
registry,
name,
reference: reference.to_string(),
}
}
/// Parse a list of image references from a newline-separated string.
pub fn parse_images_file(content: &str) -> Vec<ImageRef> {
content
.lines()
.map(|l| l.trim())
.filter(|l| !l.is_empty() && !l.starts_with('#'))
.map(parse_image_ref)
.collect()
}
/// Mirror Docker images from upstream registries into NORA.
pub async fn run_docker_mirror(
client: &Client,
nora_url: &str,
images: &[ImageRef],
concurrency: usize,
) -> Result<MirrorResult, String> {
let docker_auth = DockerAuth::new(DEFAULT_TIMEOUT);
let semaphore = std::sync::Arc::new(tokio::sync::Semaphore::new(concurrency));
let pb = create_progress_bar(images.len() as u64);
let nora_base = nora_url.trim_end_matches('/');
let mut total_fetched = 0usize;
let mut total_failed = 0usize;
let mut total_bytes = 0u64;
for image in images {
let _permit = semaphore.acquire().await.map_err(|e| e.to_string())?;
pb.set_message(format!("{}:{}", image.name, image.reference));
match mirror_single_image(client, nora_base, image, &docker_auth).await {
Ok(bytes) => {
total_fetched += 1;
total_bytes += bytes;
}
Err(e) => {
tracing::warn!(
image = %format!("{}/{}:{}", image.registry, image.name, image.reference),
error = %e,
"Failed to mirror image"
);
total_failed += 1;
}
}
pb.inc(1);
}
pb.finish_with_message("done");
Ok(MirrorResult {
total: images.len(),
fetched: total_fetched,
failed: total_failed,
bytes: total_bytes,
})
}
/// Mirror a single image: fetch manifest + blobs from upstream, push to NORA.
async fn mirror_single_image(
client: &Client,
nora_base: &str,
image: &ImageRef,
docker_auth: &DockerAuth,
) -> Result<u64, String> {
let mut bytes = 0u64;
// 1. Fetch manifest from upstream
let (manifest_bytes, content_type) = crate::registry::docker::fetch_manifest_from_upstream(
client,
&image.registry,
&image.name,
&image.reference,
docker_auth,
DEFAULT_TIMEOUT,
None,
)
.await
.map_err(|()| format!("Failed to fetch manifest for {}", image.name))?;
bytes += manifest_bytes.len() as u64;
// 2. Parse manifest to find layer digests
let manifest_json: serde_json::Value = serde_json::from_slice(&manifest_bytes)
.map_err(|e| format!("Invalid manifest JSON: {}", e))?;
// Check if this is a manifest list / OCI index
let manifests_to_process = if is_manifest_list(&content_type, &manifest_json) {
// Pick linux/amd64 manifest from the list
resolve_platform_manifest(
client,
&image.registry,
&image.name,
docker_auth,
&manifest_json,
)
.await?
} else {
vec![(
manifest_bytes.clone(),
manifest_json.clone(),
content_type.clone(),
)]
};
for (mf_bytes, mf_json, mf_ct) in &manifests_to_process {
// 3. Get config digest and layer digests
let blobs = extract_blob_digests(mf_json);
// 4. For each blob, check if NORA already has it, otherwise fetch and push
for digest in &blobs {
if blob_exists(client, nora_base, &image.name, digest).await {
tracing::debug!(digest = %digest, "Blob already exists, skipping");
continue;
}
let blob_data = crate::registry::docker::fetch_blob_from_upstream(
client,
&image.registry,
&image.name,
digest,
docker_auth,
DEFAULT_TIMEOUT,
None,
)
.await
.map_err(|()| format!("Failed to fetch blob {}", digest))?;
bytes += blob_data.len() as u64;
push_blob(client, nora_base, &image.name, digest, &blob_data).await?;
}
// 5. Push manifest to NORA
push_manifest(
client,
nora_base,
&image.name,
&image.reference,
mf_bytes,
mf_ct,
)
.await?;
}
// If this was a manifest list, also push the list itself
if manifests_to_process.len() > 1 || is_manifest_list(&content_type, &manifest_json) {
push_manifest(
client,
nora_base,
&image.name,
&image.reference,
&manifest_bytes,
&content_type,
)
.await?;
}
Ok(bytes)
}
/// Check if a manifest is a manifest list (fat manifest) or OCI index.
fn is_manifest_list(content_type: &str, json: &serde_json::Value) -> bool {
content_type.contains("manifest.list")
|| content_type.contains("image.index")
|| json.get("manifests").is_some()
}
/// From a manifest list, resolve the linux/amd64 platform manifest.
async fn resolve_platform_manifest(
client: &Client,
upstream_url: &str,
name: &str,
docker_auth: &DockerAuth,
list_json: &serde_json::Value,
) -> Result<Vec<(Vec<u8>, serde_json::Value, String)>, String> {
let manifests = list_json
.get("manifests")
.and_then(|m| m.as_array())
.ok_or("Manifest list has no manifests array")?;
// Find linux/amd64 manifest
let target = manifests
.iter()
.find(|m| {
let platform = m.get("platform");
let os = platform
.and_then(|p| p.get("os"))
.and_then(|v| v.as_str())
.unwrap_or("");
let arch = platform
.and_then(|p| p.get("architecture"))
.and_then(|v| v.as_str())
.unwrap_or("");
os == "linux" && arch == "amd64"
})
.or_else(|| manifests.first())
.ok_or("No suitable platform manifest found")?;
let digest = target
.get("digest")
.and_then(|d| d.as_str())
.ok_or("Manifest entry missing digest")?;
let (mf_bytes, mf_ct) = crate::registry::docker::fetch_manifest_from_upstream(
client,
upstream_url,
name,
digest,
docker_auth,
DEFAULT_TIMEOUT,
None,
)
.await
.map_err(|()| format!("Failed to fetch platform manifest {}", digest))?;
let mf_json: serde_json::Value = serde_json::from_slice(&mf_bytes)
.map_err(|e| format!("Invalid platform manifest: {}", e))?;
Ok(vec![(mf_bytes, mf_json, mf_ct)])
}
/// Extract all blob digests from a manifest (config + layers).
fn extract_blob_digests(manifest: &serde_json::Value) -> Vec<String> {
let mut digests = Vec::new();
// Config blob
if let Some(digest) = manifest
.get("config")
.and_then(|c| c.get("digest"))
.and_then(|d| d.as_str())
{
digests.push(digest.to_string());
}
// Layer blobs
if let Some(layers) = manifest.get("layers").and_then(|l| l.as_array()) {
for layer in layers {
if let Some(digest) = layer.get("digest").and_then(|d| d.as_str()) {
digests.push(digest.to_string());
}
}
}
digests
}
/// Check if NORA already has a blob via HEAD request.
async fn blob_exists(client: &Client, nora_base: &str, name: &str, digest: &str) -> bool {
let url = format!("{}/v2/{}/blobs/{}", nora_base, name, digest);
matches!(
client
.head(&url)
.timeout(Duration::from_secs(10))
.send()
.await,
Ok(r) if r.status().is_success()
)
}
/// Push a blob to NORA via monolithic upload.
async fn push_blob(
client: &Client,
nora_base: &str,
name: &str,
digest: &str,
data: &[u8],
) -> Result<(), String> {
// Start upload session
let start_url = format!("{}/v2/{}/blobs/uploads/", nora_base, name);
let response = client
.post(&start_url)
.timeout(Duration::from_secs(30))
.send()
.await
.map_err(|e| format!("Failed to start blob upload: {}", e))?;
let location = response
.headers()
.get("location")
.and_then(|v| v.to_str().ok())
.ok_or("Missing Location header from upload start")?
.to_string();
// Complete upload with digest
let upload_url = if location.contains('?') {
format!("{}&digest={}", location, digest)
} else {
format!("{}?digest={}", location, digest)
};
// Make absolute URL if relative
let upload_url = if upload_url.starts_with('/') {
format!("{}{}", nora_base, upload_url)
} else {
upload_url
};
let resp = client
.put(&upload_url)
.header("Content-Type", "application/octet-stream")
.body(data.to_vec())
.timeout(Duration::from_secs(DEFAULT_TIMEOUT))
.send()
.await
.map_err(|e| format!("Failed to upload blob: {}", e))?;
if !resp.status().is_success() && resp.status().as_u16() != 201 {
return Err(format!("Blob upload failed with status {}", resp.status()));
}
Ok(())
}
/// Push a manifest to NORA.
async fn push_manifest(
client: &Client,
nora_base: &str,
name: &str,
reference: &str,
data: &[u8],
content_type: &str,
) -> Result<(), String> {
let url = format!("{}/v2/{}/manifests/{}", nora_base, name, reference);
let resp = client
.put(&url)
.header("Content-Type", content_type)
.body(data.to_vec())
.timeout(Duration::from_secs(30))
.send()
.await
.map_err(|e| format!("Failed to push manifest: {}", e))?;
if !resp.status().is_success() && resp.status().as_u16() != 201 {
return Err(format!(
"Manifest push failed with status {}",
resp.status()
));
}
Ok(())
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
// --- parse_image_ref tests ---
#[test]
fn test_parse_simple_name() {
let r = parse_image_ref("alpine");
assert_eq!(r.registry, DEFAULT_REGISTRY);
assert_eq!(r.name, "library/alpine");
assert_eq!(r.reference, "latest");
}
#[test]
fn test_parse_name_with_tag() {
let r = parse_image_ref("alpine:3.20");
assert_eq!(r.registry, DEFAULT_REGISTRY);
assert_eq!(r.name, "library/alpine");
assert_eq!(r.reference, "3.20");
}
#[test]
fn test_parse_org_image() {
let r = parse_image_ref("grafana/grafana:latest");
assert_eq!(r.registry, DEFAULT_REGISTRY);
assert_eq!(r.name, "grafana/grafana");
assert_eq!(r.reference, "latest");
}
#[test]
fn test_parse_org_image_no_tag() {
let r = parse_image_ref("grafana/grafana");
assert_eq!(r.registry, DEFAULT_REGISTRY);
assert_eq!(r.name, "grafana/grafana");
assert_eq!(r.reference, "latest");
}
#[test]
fn test_parse_custom_registry() {
let r = parse_image_ref("ghcr.io/owner/repo:v1.0");
assert_eq!(r.registry, "https://ghcr.io");
assert_eq!(r.name, "owner/repo");
assert_eq!(r.reference, "v1.0");
}
#[test]
fn test_parse_digest_reference() {
let r = parse_image_ref("alpine@sha256:abcdef1234567890");
assert_eq!(r.registry, DEFAULT_REGISTRY);
assert_eq!(r.name, "library/alpine");
assert_eq!(r.reference, "sha256:abcdef1234567890");
}
#[test]
fn test_parse_registry_with_port() {
let r = parse_image_ref("localhost:5000/myimage:v1");
assert_eq!(r.registry, "https://localhost:5000");
assert_eq!(r.name, "myimage");
assert_eq!(r.reference, "v1");
}
#[test]
fn test_parse_deep_path() {
let r = parse_image_ref("ghcr.io/org/sub/image:latest");
assert_eq!(r.registry, "https://ghcr.io");
assert_eq!(r.name, "org/sub/image");
assert_eq!(r.reference, "latest");
}
#[test]
fn test_parse_trimmed() {
let r = parse_image_ref(" alpine:3.20 ");
assert_eq!(r.name, "library/alpine");
assert_eq!(r.reference, "3.20");
}
#[test]
fn test_parse_images_file() {
let content = "alpine:3.20\n# comment\npostgres:15\n\nnginx:1.25\n";
let images = parse_images_file(content);
assert_eq!(images.len(), 3);
assert_eq!(images[0].name, "library/alpine");
assert_eq!(images[1].name, "library/postgres");
assert_eq!(images[2].name, "library/nginx");
}
#[test]
fn test_parse_images_file_empty() {
let images = parse_images_file("");
assert!(images.is_empty());
}
#[test]
fn test_parse_images_file_comments_only() {
let images = parse_images_file("# comment\n# another\n");
assert!(images.is_empty());
}
// --- extract_blob_digests tests ---
#[test]
fn test_extract_blob_digests_full_manifest() {
let manifest = serde_json::json!({
"config": {
"digest": "sha256:config111"
},
"layers": [
{"digest": "sha256:layer111"},
{"digest": "sha256:layer222"}
]
});
let digests = extract_blob_digests(&manifest);
assert_eq!(digests.len(), 3);
assert_eq!(digests[0], "sha256:config111");
assert_eq!(digests[1], "sha256:layer111");
assert_eq!(digests[2], "sha256:layer222");
}
#[test]
fn test_extract_blob_digests_no_layers() {
let manifest = serde_json::json!({
"config": { "digest": "sha256:config111" }
});
let digests = extract_blob_digests(&manifest);
assert_eq!(digests.len(), 1);
}
#[test]
fn test_extract_blob_digests_empty() {
let manifest = serde_json::json!({});
let digests = extract_blob_digests(&manifest);
assert!(digests.is_empty());
}
// --- is_manifest_list tests ---
#[test]
fn test_is_manifest_list_by_content_type() {
let json = serde_json::json!({});
assert!(is_manifest_list(
"application/vnd.docker.distribution.manifest.list.v2+json",
&json
));
}
#[test]
fn test_is_manifest_list_oci_index() {
let json = serde_json::json!({});
assert!(is_manifest_list(
"application/vnd.oci.image.index.v1+json",
&json
));
}
#[test]
fn test_is_manifest_list_by_manifests_key() {
let json = serde_json::json!({
"manifests": [{"digest": "sha256:abc"}]
});
assert!(is_manifest_list(
"application/vnd.docker.distribution.manifest.v2+json",
&json
));
}
#[test]
fn test_is_not_manifest_list() {
let json = serde_json::json!({
"config": {},
"layers": []
});
assert!(!is_manifest_list(
"application/vnd.docker.distribution.manifest.v2+json",
&json
));
}
}

View File

@@ -1,567 +0,0 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT
//! `nora mirror` — pre-fetch dependencies through NORA proxy cache.
mod docker;
mod npm;
use clap::Subcommand;
use indicatif::{ProgressBar, ProgressStyle};
use std::path::PathBuf;
use std::time::Instant;
#[derive(Subcommand)]
pub enum MirrorFormat {
/// Mirror npm packages
Npm {
/// Path to package-lock.json (v1/v2/v3)
#[arg(long, conflicts_with = "packages")]
lockfile: Option<PathBuf>,
/// Comma-separated package names
#[arg(long, conflicts_with = "lockfile", value_delimiter = ',')]
packages: Option<Vec<String>>,
/// Fetch all versions (only with --packages)
#[arg(long)]
all_versions: bool,
},
/// Mirror npm packages from yarn.lock
Yarn {
/// Path to yarn.lock
#[arg(long)]
lockfile: PathBuf,
},
/// Mirror Python packages
Pip {
/// Path to requirements.txt
#[arg(long)]
lockfile: PathBuf,
},
/// Mirror Cargo crates
Cargo {
/// Path to Cargo.lock
#[arg(long)]
lockfile: PathBuf,
},
/// Mirror Maven artifacts
Maven {
/// Path to dependency list (mvn dependency:list output)
#[arg(long)]
lockfile: PathBuf,
},
/// Mirror Docker images from upstream registries
Docker {
/// Comma-separated image references (e.g., alpine:3.20,postgres:15)
#[arg(long, conflicts_with = "images_file", value_delimiter = ',')]
images: Option<Vec<String>>,
/// Path to file with image references (one per line)
#[arg(long, conflicts_with = "images")]
images_file: Option<PathBuf>,
},
}
#[derive(Debug, Clone, Hash, Eq, PartialEq)]
pub struct MirrorTarget {
pub name: String,
pub version: String,
}
#[derive(serde::Serialize)]
pub struct MirrorResult {
pub total: usize,
pub fetched: usize,
pub failed: usize,
pub bytes: u64,
}
pub fn create_progress_bar(total: u64) -> ProgressBar {
let pb = ProgressBar::new(total);
pb.set_style(
ProgressStyle::default_bar()
.template(
"{spinner:.green} [{elapsed_precise}] [{bar:40.cyan/blue}] {pos}/{len} ({eta}) {msg}",
)
.expect("static progress bar template is valid")
.progress_chars("=>-"),
);
pb
}
pub async fn run_mirror(
format: MirrorFormat,
registry: &str,
concurrency: usize,
json_output: bool,
) -> Result<(), String> {
let client = reqwest::Client::builder()
.timeout(std::time::Duration::from_secs(120))
.build()
.map_err(|e| format!("Failed to create HTTP client: {}", e))?;
// Health check
let health_url = format!("{}/health", registry.trim_end_matches('/'));
match client.get(&health_url).send().await {
Ok(r) if r.status().is_success() => {}
_ => {
return Err(format!(
"Cannot connect to NORA at {}. Is `nora serve` running?",
registry
))
}
}
let start = Instant::now();
let result = match format {
MirrorFormat::Npm {
lockfile,
packages,
all_versions,
} => {
if let Some(ref lf) = lockfile {
let content = std::fs::read_to_string(lf)
.map_err(|e| format!("Cannot read {}: {}", lf.display(), e))?;
if content.contains("# yarn lockfile v1")
|| content.starts_with("# THIS IS AN AUTOGENERATED FILE")
{
return Err("This looks like a yarn.lock file. Use `nora mirror yarn --lockfile` instead.".to_string());
}
}
npm::run_npm_mirror(
&client,
registry,
lockfile,
packages,
all_versions,
concurrency,
)
.await?
}
MirrorFormat::Yarn { lockfile } => {
let content = std::fs::read_to_string(&lockfile)
.map_err(|e| format!("Cannot read {}: {}", lockfile.display(), e))?;
let targets = npm::parse_yarn_lock(&content);
if targets.is_empty() {
println!("No packages found in {}", lockfile.display());
MirrorResult {
total: 0,
fetched: 0,
failed: 0,
bytes: 0,
}
} else {
println!(
"Mirroring {} npm packages from yarn.lock via {}...",
targets.len(),
registry
);
npm::mirror_npm_packages(&client, registry, &targets, concurrency).await?
}
}
MirrorFormat::Pip { lockfile } => {
mirror_lockfile(&client, registry, "pip", &lockfile).await?
}
MirrorFormat::Cargo { lockfile } => {
mirror_lockfile(&client, registry, "cargo", &lockfile).await?
}
MirrorFormat::Maven { lockfile } => {
mirror_lockfile(&client, registry, "maven", &lockfile).await?
}
MirrorFormat::Docker {
images,
images_file,
} => {
let image_refs = if let Some(file) = images_file {
let content = std::fs::read_to_string(&file)
.map_err(|e| format!("Cannot read {}: {}", file.display(), e))?;
docker::parse_images_file(&content)
} else if let Some(imgs) = images {
imgs.iter().map(|s| docker::parse_image_ref(s)).collect()
} else {
return Err("Either --images or --images-file is required".to_string());
};
if image_refs.is_empty() {
return Err("No images specified".to_string());
}
println!(
"Mirroring {} Docker images via {}...",
image_refs.len(),
registry
);
docker::run_docker_mirror(&client, registry, &image_refs, concurrency).await?
}
};
let elapsed = start.elapsed();
if json_output {
println!(
"{}",
serde_json::to_string_pretty(&result).unwrap_or_default()
);
} else {
println!("\nMirror complete:");
println!(" Total: {}", result.total);
println!(" Fetched: {}", result.fetched);
println!(" Failed: {}", result.failed);
println!(" Size: {:.1} MB", result.bytes as f64 / 1_048_576.0);
println!(" Time: {:.1}s", elapsed.as_secs_f64());
}
if result.failed > 0 {
Err(format!("{} packages failed to mirror", result.failed))
} else {
Ok(())
}
}
async fn mirror_lockfile(
client: &reqwest::Client,
registry: &str,
format: &str,
lockfile: &PathBuf,
) -> Result<MirrorResult, String> {
let content = std::fs::read_to_string(lockfile)
.map_err(|e| format!("Cannot read {}: {}", lockfile.display(), e))?;
let targets = match format {
"pip" => parse_requirements_txt(&content),
"cargo" => parse_cargo_lock(&content)?,
"maven" => parse_maven_deps(&content),
_ => vec![],
};
if targets.is_empty() {
println!("No packages found in {}", lockfile.display());
return Ok(MirrorResult {
total: 0,
fetched: 0,
failed: 0,
bytes: 0,
});
}
let pb = create_progress_bar(targets.len() as u64);
let base = registry.trim_end_matches('/');
let mut fetched = 0;
let mut failed = 0;
let mut bytes = 0u64;
for target in &targets {
let url = match format {
"pip" => format!("{}/simple/{}/", base, target.name),
"cargo" => format!(
"{}/cargo/api/v1/crates/{}/{}/download",
base, target.name, target.version
),
"maven" => {
let parts: Vec<&str> = target.name.split(':').collect();
if parts.len() == 2 {
let group_path = parts[0].replace('.', "/");
format!(
"{}/maven2/{}/{}/{}/{}-{}.jar",
base, group_path, parts[1], target.version, parts[1], target.version
)
} else {
pb.inc(1);
failed += 1;
continue;
}
}
_ => continue,
};
match client.get(&url).send().await {
Ok(r) if r.status().is_success() => {
if let Ok(body) = r.bytes().await {
bytes += body.len() as u64;
}
fetched += 1;
}
Ok(r) => {
eprintln!(
" WARN: {} {} -> HTTP {}",
target.name,
target.version,
r.status()
);
failed += 1;
}
Err(e) => {
eprintln!(" WARN: {} {} -> {}", target.name, target.version, e);
failed += 1;
}
}
pb.set_message(format!("{}@{}", target.name, target.version));
pb.inc(1);
}
pb.finish_with_message("done");
if format == "pip" && fetched > 0 {
eprintln!(
" NOTE: Only top-level packages were mirrored. For air-gapped installs,\n use `pip freeze > requirements.txt` to include all transitive dependencies."
);
}
Ok(MirrorResult {
total: targets.len(),
fetched,
failed,
bytes,
})
}
fn parse_requirements_txt(content: &str) -> Vec<MirrorTarget> {
content
.lines()
.filter(|l| !l.trim().is_empty() && !l.starts_with('#') && !l.starts_with('-'))
.filter_map(|line| {
let line = line.split('#').next().unwrap_or(line).trim();
if let Some((name, version)) = line.split_once("==") {
Some(MirrorTarget {
name: name.trim().to_string(),
version: version.trim().to_string(),
})
} else {
let name = line.split(['>', '<', '=', '!', '~', ';']).next()?.trim();
if name.is_empty() {
None
} else {
Some(MirrorTarget {
name: name.to_string(),
version: "latest".to_string(),
})
}
}
})
.collect()
}
fn parse_cargo_lock(content: &str) -> Result<Vec<MirrorTarget>, String> {
let lock: toml::Value =
toml::from_str(content).map_err(|e| format!("Invalid Cargo.lock: {}", e))?;
let packages = lock
.get("package")
.and_then(|p| p.as_array())
.cloned()
.unwrap_or_default();
Ok(packages
.iter()
.filter(|p| {
p.get("source")
.and_then(|s| s.as_str())
.map(|s| s.starts_with("registry+"))
.unwrap_or(false)
})
.filter_map(|p| {
let name = p.get("name")?.as_str()?.to_string();
let version = p.get("version")?.as_str()?.to_string();
Some(MirrorTarget { name, version })
})
.collect())
}
fn parse_maven_deps(content: &str) -> Vec<MirrorTarget> {
content
.lines()
.filter_map(|line| {
let line = line.trim().trim_start_matches("[INFO]").trim();
let parts: Vec<&str> = line.split(':').collect();
if parts.len() >= 4 {
let name = format!("{}:{}", parts[0], parts[1]);
let version = parts[3].to_string();
Some(MirrorTarget { name, version })
} else {
None
}
})
.collect()
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
#[test]
fn test_parse_requirements_txt() {
let content = "flask==2.3.0\nrequests>=2.28.0\n# comment\nnumpy==1.24.3\n";
let targets = parse_requirements_txt(content);
assert_eq!(targets.len(), 3);
assert_eq!(targets[0].name, "flask");
assert_eq!(targets[0].version, "2.3.0");
assert_eq!(targets[1].name, "requests");
assert_eq!(targets[1].version, "latest");
}
#[test]
fn test_parse_cargo_lock() {
let content = "\
[[package]]
name = \"serde\"
version = \"1.0.197\"
source = \"registry+https://github.com/rust-lang/crates.io-index\"
[[package]]
name = \"my-local-crate\"
version = \"0.1.0\"
";
let targets = parse_cargo_lock(content).unwrap();
assert_eq!(targets.len(), 1);
assert_eq!(targets[0].name, "serde");
}
#[test]
fn test_parse_maven_deps() {
let content = "[INFO] org.apache.commons:commons-lang3:jar:3.12.0:compile\n";
let targets = parse_maven_deps(content);
assert_eq!(targets.len(), 1);
assert_eq!(targets[0].name, "org.apache.commons:commons-lang3");
assert_eq!(targets[0].version, "3.12.0");
}
#[test]
fn test_parse_requirements_txt_empty() {
let targets = parse_requirements_txt("");
assert!(targets.is_empty());
}
#[test]
fn test_parse_requirements_txt_comments_only() {
let content = "# This is a comment\n# Another comment\n\n";
let targets = parse_requirements_txt(content);
assert!(targets.is_empty());
}
#[test]
fn test_parse_requirements_txt_flags() {
let content = "-r other-requirements.txt\n-i https://pypi.org/simple\nflask==2.0\n";
let targets = parse_requirements_txt(content);
assert_eq!(targets.len(), 1);
assert_eq!(targets[0].name, "flask");
}
#[test]
fn test_parse_requirements_txt_version_specifiers() {
let content =
"pkg1>=1.0\npkg2<2.0\npkg3!=1.5\npkg4~=1.0\npkg5==1.0 ; python_version>='3.8'\n";
let targets = parse_requirements_txt(content);
assert_eq!(targets.len(), 5);
assert_eq!(targets[0].name, "pkg1");
assert_eq!(targets[0].version, "latest");
assert_eq!(targets[4].name, "pkg5");
assert_eq!(targets[4].version, "1.0 ; python_version>='3.8'");
}
#[test]
fn test_parse_requirements_txt_inline_comments() {
let content = "flask==2.0 # web framework\n";
let targets = parse_requirements_txt(content);
assert_eq!(targets.len(), 1);
assert_eq!(targets[0].name, "flask");
assert_eq!(targets[0].version, "2.0");
}
#[test]
fn test_parse_cargo_lock_empty() {
let content = "";
let result = parse_cargo_lock(content);
let targets = result.unwrap();
assert!(targets.is_empty());
}
#[test]
fn test_parse_cargo_lock_no_packages() {
let content = "[metadata]\nsome = \"value\"\n";
let targets = parse_cargo_lock(content).unwrap();
assert!(targets.is_empty());
}
#[test]
fn test_parse_cargo_lock_git_source() {
let content = r#"
[[package]]
name = "my-dep"
version = "0.1.0"
source = "git+https://github.com/user/repo#abc123"
"#;
let targets = parse_cargo_lock(content).unwrap();
assert!(targets.is_empty()); // git sources filtered out
}
#[test]
fn test_parse_cargo_lock_multiple() {
let content = r#"
[[package]]
name = "serde"
version = "1.0.197"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "tokio"
version = "1.36.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
[[package]]
name = "local-crate"
version = "0.1.0"
"#;
let targets = parse_cargo_lock(content).unwrap();
assert_eq!(targets.len(), 2);
}
#[test]
fn test_parse_maven_deps_empty() {
let targets = parse_maven_deps("");
assert!(targets.is_empty());
}
#[test]
fn test_parse_maven_deps_short_line() {
let targets = parse_maven_deps("foo:bar\n");
assert!(targets.is_empty());
}
#[test]
fn test_parse_maven_deps_multiple() {
let content = "[INFO] org.slf4j:slf4j-api:jar:2.0.9:compile\n[INFO] com.google.guava:guava:jar:33.0.0-jre:compile\n";
let targets = parse_maven_deps(content);
assert_eq!(targets.len(), 2);
assert_eq!(targets[0].name, "org.slf4j:slf4j-api");
assert_eq!(targets[1].version, "33.0.0-jre");
}
#[test]
fn test_create_progress_bar() {
let pb = create_progress_bar(100);
assert_eq!(pb.length(), Some(100));
}
#[test]
fn test_mirror_result_json_serialization() {
let result = MirrorResult {
total: 10,
fetched: 8,
failed: 2,
bytes: 1048576,
};
let json = serde_json::to_string_pretty(&result).unwrap();
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["total"], 10);
assert_eq!(parsed["fetched"], 8);
assert_eq!(parsed["failed"], 2);
assert_eq!(parsed["bytes"], 1048576);
}
#[test]
fn test_mirror_result_json_zero_values() {
let result = MirrorResult {
total: 0,
fetched: 0,
failed: 0,
bytes: 0,
};
let json = serde_json::to_string(&result).unwrap();
assert!(json.contains("\"total\":0"));
}
}

View File

@@ -1,614 +0,0 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT
//! npm lockfile parser + mirror logic.
use super::{create_progress_bar, MirrorResult, MirrorTarget};
use std::collections::HashSet;
use std::path::PathBuf;
use tokio::sync::Semaphore;
/// Entry point for npm mirroring
pub async fn run_npm_mirror(
client: &reqwest::Client,
registry: &str,
lockfile: Option<PathBuf>,
packages: Option<Vec<String>>,
all_versions: bool,
concurrency: usize,
) -> Result<MirrorResult, String> {
let targets = if let Some(path) = lockfile {
let content = std::fs::read_to_string(&path)
.map_err(|e| format!("Cannot read {}: {}", path.display(), e))?;
parse_npm_lockfile(&content)?
} else if let Some(names) = packages {
resolve_npm_packages(client, registry, &names, all_versions).await?
} else {
return Err("Specify --lockfile or --packages".to_string());
};
if targets.is_empty() {
println!("No npm packages to mirror");
return Ok(MirrorResult {
total: 0,
fetched: 0,
failed: 0,
bytes: 0,
});
}
println!(
"Mirroring {} npm packages via {}...",
targets.len(),
registry
);
mirror_npm_packages(client, registry, &targets, concurrency).await
}
/// Parse package-lock.json (v1, v2, v3)
fn parse_npm_lockfile(content: &str) -> Result<Vec<MirrorTarget>, String> {
let json: serde_json::Value =
serde_json::from_str(content).map_err(|e| format!("Invalid JSON: {}", e))?;
let version = json
.get("lockfileVersion")
.and_then(|v| v.as_u64())
.unwrap_or(1);
let mut seen = HashSet::new();
let mut targets = Vec::new();
if version >= 2 {
// v2/v3: use "packages" object
if let Some(packages) = json.get("packages").and_then(|p| p.as_object()) {
for (key, pkg) in packages {
if key.is_empty() {
continue; // root package
}
if let Some(name) = extract_package_name(key) {
if let Some(ver) = pkg.get("version").and_then(|v| v.as_str()) {
let pair = (name.to_string(), ver.to_string());
if seen.insert(pair.clone()) {
targets.push(MirrorTarget {
name: pair.0,
version: pair.1,
});
}
}
}
}
}
}
if version == 1 || targets.is_empty() {
// v1 fallback: recursive "dependencies"
if let Some(deps) = json.get("dependencies").and_then(|d| d.as_object()) {
parse_v1_deps(deps, &mut targets, &mut seen);
}
}
Ok(targets)
}
/// Extract package name from lockfile key like "node_modules/@babel/core"
fn extract_package_name(key: &str) -> Option<&str> {
// Handle nested: "node_modules/foo/node_modules/@scope/bar" → "@scope/bar"
let last_nm = key.rfind("node_modules/")?;
let after = &key[last_nm + "node_modules/".len()..];
let name = after.trim_end_matches('/');
if name.is_empty() {
None
} else {
Some(name)
}
}
/// Recursively parse v1 lockfile "dependencies"
fn parse_v1_deps(
deps: &serde_json::Map<String, serde_json::Value>,
targets: &mut Vec<MirrorTarget>,
seen: &mut HashSet<(String, String)>,
) {
for (name, pkg) in deps {
if let Some(ver) = pkg.get("version").and_then(|v| v.as_str()) {
let pair = (name.clone(), ver.to_string());
if seen.insert(pair.clone()) {
targets.push(MirrorTarget {
name: pair.0,
version: pair.1,
});
}
}
// Recurse into nested dependencies
if let Some(nested) = pkg.get("dependencies").and_then(|d| d.as_object()) {
parse_v1_deps(nested, targets, seen);
}
}
}
/// Resolve --packages list by fetching metadata from NORA
async fn resolve_npm_packages(
client: &reqwest::Client,
registry: &str,
names: &[String],
all_versions: bool,
) -> Result<Vec<MirrorTarget>, String> {
let base = registry.trim_end_matches('/');
let mut targets = Vec::new();
for name in names {
let url = format!("{}/npm/{}", base, name);
let resp = client.get(&url).send().await.map_err(|e| e.to_string())?;
if !resp.status().is_success() {
eprintln!("Warning: {} not found (HTTP {})", name, resp.status());
continue;
}
let json: serde_json::Value = resp.json().await.map_err(|e| e.to_string())?;
if all_versions {
if let Some(versions) = json.get("versions").and_then(|v| v.as_object()) {
for ver in versions.keys() {
targets.push(MirrorTarget {
name: name.clone(),
version: ver.clone(),
});
}
}
} else {
// Just latest
let latest = json
.get("dist-tags")
.and_then(|d| d.get("latest"))
.and_then(|v| v.as_str())
.unwrap_or("latest");
targets.push(MirrorTarget {
name: name.clone(),
version: latest.to_string(),
});
}
}
Ok(targets)
}
/// Fetch packages through NORA (triggers proxy cache)
pub async fn mirror_npm_packages(
client: &reqwest::Client,
registry: &str,
targets: &[MirrorTarget],
concurrency: usize,
) -> Result<MirrorResult, String> {
let base = registry.trim_end_matches('/');
let pb = create_progress_bar(targets.len() as u64);
let sem = std::sync::Arc::new(Semaphore::new(concurrency));
// Deduplicate metadata fetches (one per package name)
let unique_names: HashSet<&str> = targets.iter().map(|t| t.name.as_str()).collect();
pb.set_message("fetching metadata...");
for name in &unique_names {
let url = format!("{}/npm/{}", base, name);
let _ = client.get(&url).send().await; // trigger metadata cache
}
// Fetch tarballs concurrently
let fetched = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let failed = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
let bytes = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(0));
let mut handles = Vec::new();
for target in targets {
let permit = sem
.clone()
.acquire_owned()
.await
.expect("semaphore closed unexpectedly");
let client = client.clone();
let pb = pb.clone();
let fetched = fetched.clone();
let failed = failed.clone();
let bytes = bytes.clone();
let short_name = target.name.split('/').next_back().unwrap_or(&target.name);
let tarball_url = format!(
"{}/npm/{}/-/{}-{}.tgz",
base, target.name, short_name, target.version
);
let label = format!("{}@{}", target.name, target.version);
handles.push(tokio::spawn(async move {
let _permit = permit;
match client.get(&tarball_url).send().await {
Ok(r) if r.status().is_success() => {
if let Ok(body) = r.bytes().await {
bytes.fetch_add(body.len() as u64, std::sync::atomic::Ordering::Relaxed);
}
fetched.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
}
_ => {
failed.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
}
}
pb.set_message(label);
pb.inc(1);
}));
}
for h in handles {
let _ = h.await;
}
pb.finish_with_message("done");
Ok(MirrorResult {
total: targets.len(),
fetched: fetched.load(std::sync::atomic::Ordering::Relaxed),
failed: failed.load(std::sync::atomic::Ordering::Relaxed),
bytes: bytes.load(std::sync::atomic::Ordering::Relaxed),
})
}
/// Parse yarn.lock v1 format
/// Format: "package@version:\n version \"X.Y.Z\"\n resolved \"url\""
pub fn parse_yarn_lock(content: &str) -> Vec<MirrorTarget> {
let mut targets = Vec::new();
let mut seen = HashSet::new();
let mut current_name: Option<String> = None;
for line in content.lines() {
let trimmed = line.trim();
// Skip comments and empty lines
if trimmed.starts_with('#') || trimmed.is_empty() {
continue;
}
// Package header: "lodash@^4.17.21:" or "@babel/core@^7.0.0, @babel/core@^7.26.0:"
if !line.starts_with(' ') && !line.starts_with('\t') && trimmed.ends_with(':') {
let header = trimmed.trim_end_matches(':');
// Take first entry before comma (all resolve to same version)
let first = header.split(',').next().unwrap_or(header).trim();
// Remove quotes if present
let first = first.trim_matches('"');
// Extract package name: everything before last @
if let Some(name) = extract_yarn_package_name(first) {
current_name = Some(name.to_string());
} else {
current_name = None;
}
continue;
}
// Version line: " version "4.17.21""
if let Some(ref name) = current_name {
if trimmed.starts_with("version ") {
let ver = trimmed.trim_start_matches("version ").trim_matches('"');
let pair = (name.clone(), ver.to_string());
if seen.insert(pair.clone()) {
targets.push(MirrorTarget {
name: pair.0,
version: pair.1,
});
}
current_name = None;
}
}
}
targets
}
/// Extract package name from yarn.lock entry like "@babel/core@^7.0.0"
fn extract_yarn_package_name(entry: &str) -> Option<&str> {
if let Some(rest) = entry.strip_prefix('@') {
// Scoped: @babel/core@^7.0.0 → find second @
let after_scope = rest.find('@')?;
Some(&entry[..after_scope + 1])
} else {
// Regular: lodash@^4.17.21 → find first @
let at = entry.find('@')?;
if at == 0 {
None
} else {
Some(&entry[..at])
}
}
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
#[test]
fn test_extract_package_name() {
assert_eq!(extract_package_name("node_modules/lodash"), Some("lodash"));
assert_eq!(
extract_package_name("node_modules/@babel/core"),
Some("@babel/core")
);
assert_eq!(
extract_package_name("node_modules/foo/node_modules/bar"),
Some("bar")
);
assert_eq!(
extract_package_name("node_modules/foo/node_modules/@types/node"),
Some("@types/node")
);
assert_eq!(extract_package_name(""), None);
}
#[test]
fn test_parse_lockfile_v3() {
let content = r#"{
"lockfileVersion": 3,
"packages": {
"": { "name": "test" },
"node_modules/lodash": { "version": "4.17.21" },
"node_modules/@babel/core": { "version": "7.26.0" },
"node_modules/@babel/core/node_modules/semver": { "version": "6.3.1" }
}
}"#;
let targets = parse_npm_lockfile(content).unwrap();
assert_eq!(targets.len(), 3);
let names: HashSet<&str> = targets.iter().map(|t| t.name.as_str()).collect();
assert!(names.contains("lodash"));
assert!(names.contains("@babel/core"));
assert!(names.contains("semver"));
}
#[test]
fn test_parse_lockfile_v1() {
let content = r#"{
"lockfileVersion": 1,
"dependencies": {
"express": {
"version": "4.18.2",
"dependencies": {
"accepts": { "version": "1.3.8" }
}
}
}
}"#;
let targets = parse_npm_lockfile(content).unwrap();
assert_eq!(targets.len(), 2);
assert_eq!(targets[0].name, "express");
assert_eq!(targets[1].name, "accepts");
}
#[test]
fn test_deduplication() {
let content = r#"{
"lockfileVersion": 3,
"packages": {
"": {},
"node_modules/debug": { "version": "4.3.4" },
"node_modules/express/node_modules/debug": { "version": "4.3.4" }
}
}"#;
let targets = parse_npm_lockfile(content).unwrap();
assert_eq!(targets.len(), 1); // deduplicated
assert_eq!(targets[0].name, "debug");
}
#[test]
fn test_extract_package_name_simple() {
assert_eq!(extract_package_name("node_modules/lodash"), Some("lodash"));
}
#[test]
fn test_extract_package_name_scoped() {
assert_eq!(
extract_package_name("node_modules/@babel/core"),
Some("@babel/core")
);
}
#[test]
fn test_extract_package_name_nested() {
assert_eq!(
extract_package_name("node_modules/foo/node_modules/@scope/bar"),
Some("@scope/bar")
);
}
#[test]
fn test_extract_package_name_no_node_modules() {
assert_eq!(extract_package_name("just/a/path"), None);
}
#[test]
fn test_extract_package_name_empty_after() {
assert_eq!(extract_package_name("node_modules/"), None);
}
#[test]
fn test_parse_lockfile_v2() {
let lockfile = serde_json::json!({
"lockfileVersion": 2,
"packages": {
"": {"name": "root"},
"node_modules/express": {"version": "4.18.2"},
"node_modules/@types/node": {"version": "20.11.0"}
}
});
let targets = parse_npm_lockfile(&lockfile.to_string()).unwrap();
assert_eq!(targets.len(), 2);
}
#[test]
fn test_parse_lockfile_empty_packages() {
let lockfile = serde_json::json!({
"lockfileVersion": 3,
"packages": {}
});
let targets = parse_npm_lockfile(&lockfile.to_string()).unwrap();
assert!(targets.is_empty());
}
#[test]
fn test_parse_lockfile_invalid_json() {
let result = parse_npm_lockfile("not json at all");
assert!(result.is_err());
}
#[test]
fn test_parse_lockfile_v1_nested() {
let lockfile = serde_json::json!({
"lockfileVersion": 1,
"dependencies": {
"express": {
"version": "4.18.2",
"dependencies": {
"accepts": {"version": "1.3.8"}
}
}
}
});
let targets = parse_npm_lockfile(&lockfile.to_string()).unwrap();
assert_eq!(targets.len(), 2);
}
#[test]
fn test_parse_lockfile_v2_falls_back_to_v1() {
// v2 with empty packages should fall back to v1 dependencies
let lockfile = serde_json::json!({
"lockfileVersion": 2,
"packages": {},
"dependencies": {
"lodash": {"version": "4.17.21"}
}
});
let targets = parse_npm_lockfile(&lockfile.to_string()).unwrap();
assert_eq!(targets.len(), 1);
assert_eq!(targets[0].name, "lodash");
}
#[test]
fn test_parse_lockfile_no_version_field() {
let lockfile = serde_json::json!({
"packages": {
"node_modules/something": {"resolved": "https://example.com"}
}
});
let targets = parse_npm_lockfile(&lockfile.to_string()).unwrap();
assert!(targets.is_empty());
}
#[test]
fn test_parse_yarn_lock_basic() {
let content = r#"# yarn lockfile v1
lodash@^4.17.21:
version "4.17.21"
resolved "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz"
express@^4.18.0:
version "4.18.2"
resolved "https://registry.npmjs.org/express/-/express-4.18.2.tgz"
"#;
let targets = parse_yarn_lock(content);
assert_eq!(targets.len(), 2);
assert_eq!(targets[0].name, "lodash");
assert_eq!(targets[0].version, "4.17.21");
assert_eq!(targets[1].name, "express");
assert_eq!(targets[1].version, "4.18.2");
}
#[test]
fn test_parse_yarn_lock_scoped() {
let content = r#"
"@babel/core@^7.26.0":
version "7.26.0"
resolved "https://registry.npmjs.org/@babel/core/-/core-7.26.0.tgz"
"#;
let targets = parse_yarn_lock(content);
assert_eq!(targets.len(), 1);
assert_eq!(targets[0].name, "@babel/core");
assert_eq!(targets[0].version, "7.26.0");
}
#[test]
fn test_parse_yarn_lock_multiple_ranges() {
let content = r#"
debug@2.6.9, debug@^2.2.0:
version "2.6.9"
resolved "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz"
debug@^4.1.0, debug@^4.3.4:
version "4.3.7"
resolved "https://registry.npmjs.org/debug/-/debug-4.3.7.tgz"
"#;
let targets = parse_yarn_lock(content);
assert_eq!(targets.len(), 2);
assert_eq!(targets[0].name, "debug");
assert_eq!(targets[0].version, "2.6.9");
assert_eq!(targets[1].name, "debug");
assert_eq!(targets[1].version, "4.3.7");
}
#[test]
fn test_parse_yarn_lock_dedup() {
let content = r#"
lodash@^4.0.0:
version "4.17.21"
lodash@^4.17.0:
version "4.17.21"
"#;
let targets = parse_yarn_lock(content);
assert_eq!(targets.len(), 1); // same name+version deduped
}
#[test]
fn test_parse_yarn_lock_empty() {
let targets = parse_yarn_lock(
"# yarn lockfile v1
",
);
assert!(targets.is_empty());
}
#[test]
fn test_parse_yarn_lock_comments_only() {
let content = "# yarn lockfile v1
# comment
";
let targets = parse_yarn_lock(content);
assert!(targets.is_empty());
}
#[test]
fn test_extract_yarn_package_name_simple() {
assert_eq!(extract_yarn_package_name("lodash@^4.17.21"), Some("lodash"));
}
#[test]
fn test_extract_yarn_package_name_scoped() {
assert_eq!(
extract_yarn_package_name("@babel/core@^7.0.0"),
Some("@babel/core")
);
}
#[test]
fn test_extract_yarn_package_name_no_at() {
assert_eq!(extract_yarn_package_name("lodash"), None);
}
#[test]
fn test_parse_yarn_lock_quoted_headers() {
let content = r#"
"@types/node@^20.0.0":
version "20.11.5"
resolved "https://registry.npmjs.org/@types/node/-/node-20.11.5.tgz"
"#;
let targets = parse_yarn_lock(content);
assert_eq!(targets.len(), 1);
assert_eq!(targets[0].name, "@types/node");
assert_eq!(targets[0].version, "20.11.5");
}
}

View File

@@ -18,8 +18,8 @@ use crate::AppState;
#[openapi( #[openapi(
info( info(
title = "Nora", title = "Nora",
version = "0.5.0", version = "0.2.12",
description = "Multi-protocol package registry supporting Docker, Maven, npm, Cargo, PyPI, Go, and Raw", description = "Multi-protocol package registry supporting Docker, Maven, npm, Cargo, and PyPI",
license(name = "MIT"), license(name = "MIT"),
contact(name = "DevITWay", url = "https://github.com/getnora-io/nora") contact(name = "DevITWay", url = "https://github.com/getnora-io/nora")
), ),
@@ -35,8 +35,6 @@ use crate::AppState;
(name = "npm", description = "npm Registry API"), (name = "npm", description = "npm Registry API"),
(name = "cargo", description = "Cargo Registry API"), (name = "cargo", description = "Cargo Registry API"),
(name = "pypi", description = "PyPI Simple API"), (name = "pypi", description = "PyPI Simple API"),
(name = "go", description = "Go Module Proxy API"),
(name = "raw", description = "Raw File Storage API"),
(name = "auth", description = "Authentication & API Tokens") (name = "auth", description = "Authentication & API Tokens")
), ),
paths( paths(

View File

@@ -25,7 +25,7 @@ pub fn auth_rate_limiter(
.burst_size(config.auth_burst) .burst_size(config.auth_burst)
.use_headers() .use_headers()
.finish() .finish()
.expect("failed to build auth rate limiter: invalid RateLimitConfig"); .expect("Failed to build auth rate limiter");
tower_governor::GovernorLayer::new(gov_config) tower_governor::GovernorLayer::new(gov_config)
} }
@@ -46,7 +46,7 @@ pub fn upload_rate_limiter(
.burst_size(config.upload_burst) .burst_size(config.upload_burst)
.use_headers() .use_headers()
.finish() .finish()
.expect("failed to build upload rate limiter: invalid RateLimitConfig"); .expect("Failed to build upload rate limiter");
tower_governor::GovernorLayer::new(gov_config) tower_governor::GovernorLayer::new(gov_config)
} }
@@ -65,7 +65,7 @@ pub fn general_rate_limiter(
.burst_size(config.general_burst) .burst_size(config.general_burst)
.use_headers() .use_headers()
.finish() .finish()
.expect("failed to build general rate limiter: invalid RateLimitConfig"); .expect("Failed to build general rate limiter");
tower_governor::GovernorLayer::new(gov_config) tower_governor::GovernorLayer::new(gov_config)
} }

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -139,7 +139,6 @@ fn parse_www_authenticate(header: &str) -> Option<HashMap<String, String>> {
} }
#[cfg(test)] #[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests { mod tests {
use super::*; use super::*;
@@ -168,152 +167,4 @@ mod tests {
Some(&"https://ghcr.io/token".to_string()) Some(&"https://ghcr.io/token".to_string())
); );
} }
#[test]
fn test_parse_www_authenticate_no_bearer() {
assert!(parse_www_authenticate("Basic realm=\"test\"").is_none());
}
#[test]
fn test_parse_www_authenticate_empty() {
assert!(parse_www_authenticate("").is_none());
}
#[test]
fn test_parse_www_authenticate_partial() {
let header = r#"Bearer realm="https://example.com/token""#;
let params = parse_www_authenticate(header).unwrap();
assert_eq!(
params.get("realm"),
Some(&"https://example.com/token".to_string())
);
assert!(!params.contains_key("service"));
}
#[test]
fn test_docker_auth_default() {
let auth = DockerAuth::default();
assert!(auth.tokens.read().is_empty());
}
#[test]
fn test_docker_auth_new() {
let auth = DockerAuth::new(30);
assert!(auth.tokens.read().is_empty());
}
#[tokio::test]
async fn test_get_token_no_www_authenticate() {
let auth = DockerAuth::default();
let result = auth
.get_token("https://registry.example.com", "library/test", None, None)
.await;
assert!(result.is_none());
}
#[tokio::test]
async fn test_get_token_cache_hit() {
let auth = DockerAuth::default();
// Manually insert a cached token
{
let mut tokens = auth.tokens.write();
tokens.insert(
"https://registry.example.com:library/test".to_string(),
CachedToken {
token: "cached-token-123".to_string(),
expires_at: Instant::now() + Duration::from_secs(300),
},
);
}
let result = auth
.get_token("https://registry.example.com", "library/test", None, None)
.await;
assert_eq!(result, Some("cached-token-123".to_string()));
}
#[tokio::test]
async fn test_get_token_cache_expired() {
let auth = DockerAuth::default();
{
let mut tokens = auth.tokens.write();
tokens.insert(
"https://registry.example.com:library/test".to_string(),
CachedToken {
token: "expired-token".to_string(),
expires_at: Instant::now() - Duration::from_secs(1),
},
);
}
// Without www_authenticate, returns None (can't fetch new token)
let result = auth
.get_token("https://registry.example.com", "library/test", None, None)
.await;
assert!(result.is_none());
}
#[test]
fn test_parse_www_authenticate_bearer_only() {
let params = parse_www_authenticate("Bearer ").unwrap();
assert!(params.is_empty());
}
#[test]
fn test_parse_www_authenticate_missing_realm() {
let header = r#"Bearer service="registry.docker.io""#;
let params = parse_www_authenticate(header).unwrap();
assert!(params.get("realm").is_none());
assert_eq!(
params.get("service"),
Some(&"registry.docker.io".to_string())
);
}
#[test]
fn test_parse_www_authenticate_missing_service() {
let header = r#"Bearer realm="https://auth.docker.io/token""#;
let params = parse_www_authenticate(header).unwrap();
assert_eq!(
params.get("realm"),
Some(&"https://auth.docker.io/token".to_string())
);
assert!(params.get("service").is_none());
}
#[test]
fn test_parse_www_authenticate_malformed_kv() {
let header = r#"Bearer garbage,realm="https://auth.docker.io/token""#;
let params = parse_www_authenticate(header).unwrap();
assert_eq!(
params.get("realm"),
Some(&"https://auth.docker.io/token".to_string())
);
}
#[tokio::test]
async fn test_fetch_token_invalid_url() {
let auth = DockerAuth::new(1);
let result = auth
.get_token(
"https://registry.example.com",
"library/test",
Some(r#"Bearer realm="http://127.0.0.1:1/token",service="test""#),
None,
)
.await;
assert!(result.is_none());
}
#[tokio::test]
async fn test_fetch_token_missing_realm_in_header() {
let auth = DockerAuth::default();
let result = auth
.get_token(
"https://registry.example.com",
"library/test",
Some(r#"Bearer service="registry.docker.io""#),
None,
)
.await;
assert!(result.is_none());
}
} }

View File

@@ -1,523 +0,0 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT
//! Go module proxy (GOPROXY protocol).
//!
//! Implements the 5 required endpoints:
//! GET /go/{module}/@v/list — list known versions
//! GET /go/{module}/@v/{ver}.info — version metadata (JSON)
//! GET /go/{module}/@v/{ver}.mod — go.mod file
//! GET /go/{module}/@v/{ver}.zip — module zip archive
//! GET /go/{module}/@latest — latest version info
use crate::activity_log::{ActionType, ActivityEntry};
use crate::audit::AuditEntry;
use crate::registry::{proxy_fetch, proxy_fetch_text, ProxyError};
use crate::AppState;
use axum::{
extract::{Path, State},
http::{header, HeaderValue, StatusCode},
response::{IntoResponse, Response},
routing::get,
Router,
};
use percent_encoding::percent_decode;
use std::sync::Arc;
pub fn routes() -> Router<Arc<AppState>> {
Router::new().route("/go/{*path}", get(handle))
}
/// Main handler — parses the wildcard path and dispatches to the right logic.
async fn handle(State(state): State<Arc<AppState>>, Path(path): Path<String>) -> Response {
// URL-decode the path: Go client sends %21 for !, Axum wildcard may not decode it
let path = percent_decode(path.as_bytes())
.decode_utf8()
.map(|s| s.into_owned())
.unwrap_or(path);
tracing::debug!(path = %path, "Go proxy request");
// Validate path: no traversal, no null bytes
if !is_safe_path(&path) {
tracing::debug!(path = %path, "Go proxy: unsafe path");
return StatusCode::BAD_REQUEST.into_response();
}
// Split: "github.com/!azure/sdk/@v/v1.0.0.info" → module + file
let (module_encoded, file) = match split_go_path(&path) {
Some(parts) => parts,
None => {
tracing::debug!(path = %path, "Go proxy: cannot split path");
return StatusCode::NOT_FOUND.into_response();
}
};
let storage_key = format!("go/{}", path);
let content_type = content_type_for(&file);
// Mutable endpoints: @v/list and @latest can be refreshed from upstream
let is_mutable = file == "@v/list" || file == "@latest";
// Immutable: .info, .mod, .zip — once cached, never overwrite
let is_immutable = !is_mutable;
// 1. Try local cache (for immutable files, this is authoritative)
if let Ok(data) = state.storage.get(&storage_key).await {
state.metrics.record_download("go");
state.metrics.record_cache_hit();
state.activity.push(ActivityEntry::new(
ActionType::CacheHit,
format_artifact(&module_encoded, &file),
"go",
"CACHE",
));
return with_content_type(data.to_vec(), content_type);
}
// 2. Try upstream proxy
let proxy_url = match &state.config.go.proxy {
Some(url) => url.clone(),
None => return StatusCode::NOT_FOUND.into_response(),
};
// Validate module path encoding (but keep encoded for upstream — proxy.golang.org expects ! encoding)
if decode_module_path(&module_encoded).is_err() {
return StatusCode::BAD_REQUEST.into_response();
}
let upstream_url = format!(
"{}/{}",
proxy_url.trim_end_matches('/'),
format_upstream_path(&module_encoded, &file)
);
// Use longer timeout for .zip files
let timeout = if file.ends_with(".zip") {
state.config.go.proxy_timeout_zip
} else {
state.config.go.proxy_timeout
};
// Fetch: binary for .zip, text for everything else
let data = if file.ends_with(".zip") {
proxy_fetch(
&state.http_client,
&upstream_url,
timeout,
state.config.go.proxy_auth.as_deref(),
)
.await
} else {
proxy_fetch_text(
&state.http_client,
&upstream_url,
timeout,
state.config.go.proxy_auth.as_deref(),
None,
)
.await
.map(|s| s.into_bytes())
};
match data {
Ok(bytes) => {
// Enforce size limit for .zip
if file.ends_with(".zip") && bytes.len() as u64 > state.config.go.max_zip_size {
tracing::warn!(
module = module_encoded,
size = bytes.len(),
limit = state.config.go.max_zip_size,
"Go module zip exceeds size limit"
);
return StatusCode::PAYLOAD_TOO_LARGE.into_response();
}
state.metrics.record_download("go");
state.metrics.record_cache_miss();
state.activity.push(ActivityEntry::new(
ActionType::ProxyFetch,
format_artifact(&module_encoded, &file),
"go",
"PROXY",
));
state
.audit
.log(AuditEntry::new("proxy_fetch", "api", "", "go", ""));
// Background cache: immutable = put_if_absent, mutable = always overwrite
let storage = state.storage.clone();
let key = storage_key.clone();
let data_clone = bytes.clone();
tokio::spawn(async move {
if is_immutable {
// Only write if not already cached (immutability guarantee)
if storage.stat(&key).await.is_none() {
let _ = storage.put(&key, &data_clone).await;
}
} else {
let _ = storage.put(&key, &data_clone).await;
}
});
state.repo_index.invalidate("go");
with_content_type(bytes, content_type)
}
Err(ProxyError::NotFound) => StatusCode::NOT_FOUND.into_response(),
Err(e) => {
tracing::debug!(
module = module_encoded,
file = file,
error = ?e,
"Go upstream proxy error"
);
StatusCode::BAD_GATEWAY.into_response()
}
}
}
// ============================================================================
// Module path encoding/decoding
// ============================================================================
/// Decode Go module path: `!x` → `X`
///
/// Go module proxy spec requires uppercase letters to be encoded as `!`
/// followed by the lowercase letter. Raw uppercase in encoded path is invalid.
fn decode_module_path(encoded: &str) -> Result<String, ()> {
let mut result = String::with_capacity(encoded.len());
let mut chars = encoded.chars();
while let Some(c) = chars.next() {
if c == '!' {
match chars.next() {
Some(next) if next.is_ascii_lowercase() => {
result.push(next.to_ascii_uppercase());
}
_ => return Err(()),
}
} else if c.is_ascii_uppercase() {
// Raw uppercase in encoded path is invalid per spec
return Err(());
} else {
result.push(c);
}
}
Ok(result)
}
/// Encode Go module path: `X` → `!x`
#[cfg(test)]
fn encode_module_path(path: &str) -> String {
let mut result = String::with_capacity(path.len() + 8);
for c in path.chars() {
if c.is_ascii_uppercase() {
result.push('!');
result.push(c.to_ascii_lowercase());
} else {
result.push(c);
}
}
result
}
// ============================================================================
// Path parsing helpers
// ============================================================================
/// Split Go path into (encoded_module, file).
///
/// Examples:
/// "github.com/user/repo/@v/v1.0.0.info" → ("github.com/user/repo", "@v/v1.0.0.info")
/// "github.com/user/repo/v2/@v/list" → ("github.com/user/repo/v2", "@v/list")
/// "github.com/user/repo/@latest" → ("github.com/user/repo", "@latest")
fn split_go_path(path: &str) -> Option<(String, String)> {
// Try @latest first (it's simpler)
if let Some(pos) = path.rfind("/@latest") {
let module = &path[..pos];
if !module.is_empty() {
return Some((module.to_string(), "@latest".to_string()));
}
}
// Try @v/ — find the last occurrence (handles /v2/@v/ correctly)
if let Some(pos) = path.rfind("/@v/") {
let module = &path[..pos];
let file = &path[pos + 1..]; // "@v/..."
if !module.is_empty() && !file.is_empty() {
return Some((module.to_string(), file.to_string()));
}
}
None
}
/// Path validation: no traversal attacks
fn is_safe_path(path: &str) -> bool {
!path.contains("..")
&& !path.starts_with('/')
&& !path.contains("//")
&& !path.contains('\0')
&& !path.is_empty()
}
/// Content-Type for Go proxy responses
fn content_type_for(file: &str) -> &'static str {
if file.ends_with(".info") || file == "@latest" {
"application/json"
} else if file.ends_with(".zip") {
"application/zip"
} else {
// .mod, @v/list
"text/plain; charset=utf-8"
}
}
/// Build upstream URL path (uses decoded module path)
fn format_upstream_path(module_decoded: &str, file: &str) -> String {
format!("{}/{}", module_decoded, file)
}
/// Human-readable artifact name for activity log
fn format_artifact(module: &str, file: &str) -> String {
if file == "@v/list" || file == "@latest" {
format!("{} {}", module, file)
} else if let Some(version_file) = file.strip_prefix("@v/") {
// "v1.0.0.info" → "module@v1.0.0"
let version = version_file
.rsplit_once('.')
.map(|(v, _ext)| v)
.unwrap_or(version_file);
format!("{}@{}", module, version)
} else {
format!("{}/{}", module, file)
}
}
/// Build response with Content-Type header
fn with_content_type(data: Vec<u8>, content_type: &'static str) -> Response {
(
StatusCode::OK,
[(header::CONTENT_TYPE, HeaderValue::from_static(content_type))],
data,
)
.into_response()
}
// ============================================================================
// Tests
// ============================================================================
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests {
use super::*;
// ── Encoding/decoding ───────────────────────────────────────────────
#[test]
fn test_decode_azure() {
assert_eq!(
decode_module_path("github.com/!azure/sdk").unwrap(),
"github.com/Azure/sdk"
);
}
#[test]
fn test_decode_multiple_uppercase() {
assert_eq!(
decode_module_path("!google!cloud!platform/foo").unwrap(),
"GoogleCloudPlatform/foo"
);
}
#[test]
fn test_decode_no_uppercase() {
assert_eq!(
decode_module_path("github.com/user/repo").unwrap(),
"github.com/user/repo"
);
}
#[test]
fn test_decode_invalid_bang_at_end() {
assert!(decode_module_path("foo!").is_err());
}
#[test]
fn test_decode_invalid_bang_followed_by_uppercase() {
assert!(decode_module_path("foo!A").is_err());
}
#[test]
fn test_decode_raw_uppercase_is_invalid() {
assert!(decode_module_path("github.com/Azure/sdk").is_err());
}
#[test]
fn test_encode_roundtrip() {
let original = "github.com/Azure/azure-sdk-for-go";
let encoded = encode_module_path(original);
assert_eq!(encoded, "github.com/!azure/azure-sdk-for-go");
assert_eq!(decode_module_path(&encoded).unwrap(), original);
}
#[test]
fn test_encode_no_change() {
assert_eq!(
encode_module_path("github.com/user/repo"),
"github.com/user/repo"
);
}
// ── Path splitting ──────────────────────────────────────────────────
#[test]
fn test_split_version_info() {
let (module, file) = split_go_path("github.com/user/repo/@v/v1.0.0.info").unwrap();
assert_eq!(module, "github.com/user/repo");
assert_eq!(file, "@v/v1.0.0.info");
}
#[test]
fn test_split_version_list() {
let (module, file) = split_go_path("github.com/user/repo/@v/list").unwrap();
assert_eq!(module, "github.com/user/repo");
assert_eq!(file, "@v/list");
}
#[test]
fn test_split_latest() {
let (module, file) = split_go_path("github.com/user/repo/@latest").unwrap();
assert_eq!(module, "github.com/user/repo");
assert_eq!(file, "@latest");
}
#[test]
fn test_split_major_version_suffix() {
let (module, file) = split_go_path("github.com/user/repo/v2/@v/list").unwrap();
assert_eq!(module, "github.com/user/repo/v2");
assert_eq!(file, "@v/list");
}
#[test]
fn test_split_incompatible_version() {
let (module, file) =
split_go_path("github.com/user/repo/@v/v4.1.2+incompatible.info").unwrap();
assert_eq!(module, "github.com/user/repo");
assert_eq!(file, "@v/v4.1.2+incompatible.info");
}
#[test]
fn test_split_pseudo_version() {
let (module, file) =
split_go_path("github.com/user/repo/@v/v0.0.0-20210101000000-abcdef123456.info")
.unwrap();
assert_eq!(module, "github.com/user/repo");
assert_eq!(file, "@v/v0.0.0-20210101000000-abcdef123456.info");
}
#[test]
fn test_split_no_at() {
assert!(split_go_path("github.com/user/repo/v1.0.0").is_none());
}
#[test]
fn test_split_empty_module() {
assert!(split_go_path("/@v/list").is_none());
}
// ── Path safety ─────────────────────────────────────────────────────
#[test]
fn test_safe_path_normal() {
assert!(is_safe_path("github.com/user/repo/@v/list"));
}
#[test]
fn test_reject_traversal() {
assert!(!is_safe_path("../../etc/passwd"));
}
#[test]
fn test_reject_absolute() {
assert!(!is_safe_path("/etc/passwd"));
}
#[test]
fn test_reject_double_slash() {
assert!(!is_safe_path("github.com//evil/@v/list"));
}
#[test]
fn test_reject_null() {
assert!(!is_safe_path("github.com/\0evil/@v/list"));
}
#[test]
fn test_reject_empty() {
assert!(!is_safe_path(""));
}
// ── Content-Type ────────────────────────────────────────────────────
#[test]
fn test_content_type_info() {
assert_eq!(content_type_for("@v/v1.0.0.info"), "application/json");
}
#[test]
fn test_content_type_latest() {
assert_eq!(content_type_for("@latest"), "application/json");
}
#[test]
fn test_content_type_zip() {
assert_eq!(content_type_for("@v/v1.0.0.zip"), "application/zip");
}
#[test]
fn test_content_type_mod() {
assert_eq!(
content_type_for("@v/v1.0.0.mod"),
"text/plain; charset=utf-8"
);
}
#[test]
fn test_content_type_list() {
assert_eq!(content_type_for("@v/list"), "text/plain; charset=utf-8");
}
// ── Artifact formatting ─────────────────────────────────────────────
#[test]
fn test_format_artifact_version() {
assert_eq!(
format_artifact("github.com/user/repo", "@v/v1.0.0.info"),
"github.com/user/repo@v1.0.0"
);
}
#[test]
fn test_format_artifact_list() {
assert_eq!(
format_artifact("github.com/user/repo", "@v/list"),
"github.com/user/repo @v/list"
);
}
#[test]
fn test_format_artifact_latest() {
assert_eq!(
format_artifact("github.com/user/repo", "@latest"),
"github.com/user/repo @latest"
);
}
#[test]
fn test_format_artifact_zip() {
assert_eq!(
format_artifact("github.com/user/repo", "@v/v1.0.0.zip"),
"github.com/user/repo@v1.0.0"
);
}
}

View File

@@ -3,7 +3,7 @@
use crate::activity_log::{ActionType, ActivityEntry}; use crate::activity_log::{ActionType, ActivityEntry};
use crate::audit::AuditEntry; use crate::audit::AuditEntry;
use crate::registry::proxy_fetch; use crate::config::basic_auth_header;
use crate::AppState; use crate::AppState;
use axum::{ use axum::{
body::Bytes, body::Bytes,
@@ -14,6 +14,7 @@ use axum::{
Router, Router,
}; };
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration;
pub fn routes() -> Router<Arc<AppState>> { pub fn routes() -> Router<Arc<AppState>> {
Router::new() Router::new()
@@ -52,7 +53,7 @@ async fn download(State(state): State<Arc<AppState>>, Path(path): Path<String>)
for proxy in &state.config.maven.proxies { for proxy in &state.config.maven.proxies {
let url = format!("{}/{}", proxy.url().trim_end_matches('/'), path); let url = format!("{}/{}", proxy.url().trim_end_matches('/'), path);
match proxy_fetch( match fetch_from_proxy(
&state.http_client, &state.http_client,
&url, &url,
state.config.maven.proxy_timeout, state.config.maven.proxy_timeout,
@@ -127,6 +128,25 @@ async fn upload(
} }
} }
async fn fetch_from_proxy(
client: &reqwest::Client,
url: &str,
timeout_secs: u64,
auth: Option<&str>,
) -> Result<Vec<u8>, ()> {
let mut request = client.get(url).timeout(Duration::from_secs(timeout_secs));
if let Some(credentials) = auth {
request = request.header("Authorization", basic_auth_header(credentials));
}
let response = request.send().await.map_err(|_| ())?;
if !response.status().is_success() {
return Err(());
}
response.bytes().await.map(|b| b.to_vec()).map_err(|_| ())
}
fn with_content_type( fn with_content_type(
path: &str, path: &str,
data: Bytes, data: Bytes,
@@ -145,148 +165,3 @@ fn with_content_type(
(StatusCode::OK, [(header::CONTENT_TYPE, content_type)], data) (StatusCode::OK, [(header::CONTENT_TYPE, content_type)], data)
} }
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_content_type_pom() {
let (status, headers, _) =
with_content_type("com/example/1.0/example-1.0.pom", Bytes::from("data"));
assert_eq!(status, StatusCode::OK);
assert_eq!(headers[0].1, "application/xml");
}
#[test]
fn test_content_type_jar() {
let (_, headers, _) =
with_content_type("com/example/1.0/example-1.0.jar", Bytes::from("data"));
assert_eq!(headers[0].1, "application/java-archive");
}
#[test]
fn test_content_type_xml() {
let (_, headers, _) =
with_content_type("com/example/maven-metadata.xml", Bytes::from("data"));
assert_eq!(headers[0].1, "application/xml");
}
#[test]
fn test_content_type_sha1() {
let (_, headers, _) =
with_content_type("com/example/1.0/example-1.0.jar.sha1", Bytes::from("data"));
assert_eq!(headers[0].1, "text/plain");
}
#[test]
fn test_content_type_md5() {
let (_, headers, _) =
with_content_type("com/example/1.0/example-1.0.jar.md5", Bytes::from("data"));
assert_eq!(headers[0].1, "text/plain");
}
#[test]
fn test_content_type_unknown() {
let (_, headers, _) = with_content_type("some/random/file.bin", Bytes::from("data"));
assert_eq!(headers[0].1, "application/octet-stream");
}
#[test]
fn test_content_type_preserves_body() {
let body = Bytes::from("test-jar-content");
let (_, _, data) = with_content_type("test.jar", body.clone());
assert_eq!(data, body);
}
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod integration_tests {
use crate::test_helpers::{body_bytes, create_test_context, send};
use axum::body::Body;
use axum::http::{header, Method, StatusCode};
#[tokio::test]
async fn test_maven_put_get_roundtrip() {
let ctx = create_test_context();
let jar_data = b"fake-jar-content";
let put = send(
&ctx.app,
Method::PUT,
"/maven2/com/example/mylib/1.0/mylib-1.0.jar",
Body::from(&jar_data[..]),
)
.await;
assert_eq!(put.status(), StatusCode::CREATED);
let get = send(
&ctx.app,
Method::GET,
"/maven2/com/example/mylib/1.0/mylib-1.0.jar",
"",
)
.await;
assert_eq!(get.status(), StatusCode::OK);
let body = body_bytes(get).await;
assert_eq!(&body[..], jar_data);
}
#[tokio::test]
async fn test_maven_not_found_no_proxy() {
let ctx = create_test_context();
let resp = send(
&ctx.app,
Method::GET,
"/maven2/missing/artifact/1.0/artifact-1.0.jar",
"",
)
.await;
assert_eq!(resp.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn test_maven_content_type_pom() {
let ctx = create_test_context();
send(
&ctx.app,
Method::PUT,
"/maven2/com/ex/1.0/ex-1.0.pom",
Body::from("<project/>"),
)
.await;
let get = send(&ctx.app, Method::GET, "/maven2/com/ex/1.0/ex-1.0.pom", "").await;
assert_eq!(get.status(), StatusCode::OK);
assert_eq!(
get.headers().get(header::CONTENT_TYPE).unwrap(),
"application/xml"
);
}
#[tokio::test]
async fn test_maven_content_type_jar() {
let ctx = create_test_context();
send(
&ctx.app,
Method::PUT,
"/maven2/org/test/app/2.0/app-2.0.jar",
Body::from("jar-data"),
)
.await;
let get = send(
&ctx.app,
Method::GET,
"/maven2/org/test/app/2.0/app-2.0.jar",
"",
)
.await;
assert_eq!(get.status(), StatusCode::OK);
assert_eq!(
get.headers().get(header::CONTENT_TYPE).unwrap(),
"application/java-archive"
);
}
}

View File

@@ -4,7 +4,6 @@
mod cargo_registry; mod cargo_registry;
pub mod docker; pub mod docker;
pub mod docker_auth; pub mod docker_auth;
mod go;
mod maven; mod maven;
mod npm; mod npm;
mod pypi; mod pypi;
@@ -13,108 +12,7 @@ mod raw;
pub use cargo_registry::routes as cargo_routes; pub use cargo_registry::routes as cargo_routes;
pub use docker::routes as docker_routes; pub use docker::routes as docker_routes;
pub use docker_auth::DockerAuth; pub use docker_auth::DockerAuth;
pub use go::routes as go_routes;
pub use maven::routes as maven_routes; pub use maven::routes as maven_routes;
pub use npm::routes as npm_routes; pub use npm::routes as npm_routes;
pub use pypi::routes as pypi_routes; pub use pypi::routes as pypi_routes;
pub use raw::routes as raw_routes; pub use raw::routes as raw_routes;
use crate::config::basic_auth_header;
use std::time::Duration;
#[derive(Debug)]
#[allow(dead_code)]
pub(crate) enum ProxyError {
NotFound,
Upstream(u16),
Network(String),
}
/// Core fetch logic with retry. Callers provide a response extractor.
async fn proxy_fetch_core<T, F, Fut>(
client: &reqwest::Client,
url: &str,
timeout_secs: u64,
auth: Option<&str>,
extra_headers: Option<(&str, &str)>,
extract: F,
) -> Result<T, ProxyError>
where
F: Fn(reqwest::Response) -> Fut + Copy,
Fut: std::future::Future<Output = Result<T, reqwest::Error>>,
{
for attempt in 0..2 {
let mut request = client.get(url).timeout(Duration::from_secs(timeout_secs));
if let Some(credentials) = auth {
request = request.header("Authorization", basic_auth_header(credentials));
}
if let Some((key, val)) = extra_headers {
request = request.header(key, val);
}
match request.send().await {
Ok(response) => {
if response.status().is_success() {
return extract(response)
.await
.map_err(|e| ProxyError::Network(e.to_string()));
}
let status = response.status().as_u16();
if (400..500).contains(&status) {
return Err(ProxyError::NotFound);
}
if attempt == 0 {
tracing::debug!(url, status, "upstream 5xx, retrying in 1s");
tokio::time::sleep(Duration::from_secs(1)).await;
continue;
}
return Err(ProxyError::Upstream(status));
}
Err(e) => {
if attempt == 0 {
tracing::debug!(url, error = %e, "upstream error, retrying in 1s");
tokio::time::sleep(Duration::from_secs(1)).await;
continue;
}
return Err(ProxyError::Network(e.to_string()));
}
}
}
Err(ProxyError::Network("max retries exceeded".into()))
}
/// Fetch binary content from upstream proxy with timeout and 1 retry.
pub(crate) async fn proxy_fetch(
client: &reqwest::Client,
url: &str,
timeout_secs: u64,
auth: Option<&str>,
) -> Result<Vec<u8>, ProxyError> {
proxy_fetch_core(client, url, timeout_secs, auth, None, |r| async {
r.bytes().await.map(|b| b.to_vec())
})
.await
}
/// Fetch text content from upstream proxy with timeout and 1 retry.
pub(crate) async fn proxy_fetch_text(
client: &reqwest::Client,
url: &str,
timeout_secs: u64,
auth: Option<&str>,
extra_headers: Option<(&str, &str)>,
) -> Result<String, ProxyError> {
proxy_fetch_core(client, url, timeout_secs, auth, extra_headers, |r| r.text()).await
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_proxy_fetch_invalid_url() {
let client = reqwest::Client::new();
let result = proxy_fetch(&client, "http://127.0.0.1:1/nonexistent", 2, None).await;
assert!(matches!(result, Err(ProxyError::Network(_))));
}
}

View File

@@ -3,7 +3,7 @@
use crate::activity_log::{ActionType, ActivityEntry}; use crate::activity_log::{ActionType, ActivityEntry};
use crate::audit::AuditEntry; use crate::audit::AuditEntry;
use crate::registry::proxy_fetch; use crate::config::basic_auth_header;
use crate::AppState; use crate::AppState;
use axum::{ use axum::{
body::Bytes, body::Bytes,
@@ -16,6 +16,7 @@ use axum::{
use base64::Engine; use base64::Engine;
use sha2::Digest; use sha2::Digest;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration;
pub fn routes() -> Router<Arc<AppState>> { pub fn routes() -> Router<Arc<AppState>> {
Router::new() Router::new()
@@ -107,7 +108,7 @@ async fn handle_request(State(state): State<Arc<AppState>>, Path(path): Path<Str
// Tarball: integrity check if hash exists // Tarball: integrity check if hash exists
let hash_key = format!("{}.sha256", key); let hash_key = format!("{}.sha256", key);
if let Ok(stored_hash) = state.storage.get(&hash_key).await { if let Ok(stored_hash) = state.storage.get(&hash_key).await {
let computed = hex::encode(sha2::Sha256::digest(&data)); let computed = format!("{:x}", sha2::Sha256::digest(&data));
let expected = String::from_utf8_lossy(&stored_hash); let expected = String::from_utf8_lossy(&stored_hash);
if computed != expected.as_ref() { if computed != expected.as_ref() {
tracing::error!( tracing::error!(
@@ -139,7 +140,7 @@ async fn handle_request(State(state): State<Arc<AppState>>, Path(path): Path<Str
if let Some(proxy_url) = &state.config.npm.proxy { if let Some(proxy_url) = &state.config.npm.proxy {
let url = format!("{}/{}", proxy_url.trim_end_matches('/'), path); let url = format!("{}/{}", proxy_url.trim_end_matches('/'), path);
if let Ok(data) = proxy_fetch( if let Ok(data) = fetch_from_proxy(
&state.http_client, &state.http_client,
&url, &url,
state.config.npm.proxy_timeout, state.config.npm.proxy_timeout,
@@ -152,7 +153,7 @@ async fn handle_request(State(state): State<Arc<AppState>>, Path(path): Path<Str
if is_tarball { if is_tarball {
// Compute and store sha256 // Compute and store sha256
let hash = hex::encode(sha2::Sha256::digest(&data)); let hash = format!("{:x}", sha2::Sha256::digest(&data));
let hash_key = format!("{}.sha256", key); let hash_key = format!("{}.sha256", key);
let storage = state.storage.clone(); let storage = state.storage.clone();
tokio::spawn(async move { tokio::spawn(async move {
@@ -176,7 +177,8 @@ async fn handle_request(State(state): State<Arc<AppState>>, Path(path): Path<Str
} else { } else {
// Metadata: rewrite tarball URLs to point to NORA // Metadata: rewrite tarball URLs to point to NORA
let nora_base = nora_base_url(&state); let nora_base = nora_base_url(&state);
let rewritten = rewrite_tarball_urls(&data, &nora_base, proxy_url).unwrap_or(data); let rewritten = rewrite_tarball_urls(&data, &nora_base, proxy_url)
.unwrap_or_else(|_| data.clone());
data_to_cache = rewritten.clone(); data_to_cache = rewritten.clone();
data_to_serve = rewritten; data_to_serve = rewritten;
@@ -206,7 +208,7 @@ async fn refetch_metadata(state: &Arc<AppState>, path: &str, key: &str) -> Optio
let proxy_url = state.config.npm.proxy.as_ref()?; let proxy_url = state.config.npm.proxy.as_ref()?;
let url = format!("{}/{}", proxy_url.trim_end_matches('/'), path); let url = format!("{}/{}", proxy_url.trim_end_matches('/'), path);
let data = proxy_fetch( let data = fetch_from_proxy(
&state.http_client, &state.http_client,
&url, &url,
state.config.npm.proxy_timeout, state.config.npm.proxy_timeout,
@@ -216,7 +218,8 @@ async fn refetch_metadata(state: &Arc<AppState>, path: &str, key: &str) -> Optio
.ok()?; .ok()?;
let nora_base = nora_base_url(state); let nora_base = nora_base_url(state);
let rewritten = rewrite_tarball_urls(&data, &nora_base, proxy_url).unwrap_or(data); let rewritten =
rewrite_tarball_urls(&data, &nora_base, proxy_url).unwrap_or_else(|_| data.clone());
let storage = state.storage.clone(); let storage = state.storage.clone();
let key_clone = key.to_string(); let key_clone = key.to_string();
@@ -338,15 +341,13 @@ async fn handle_publish(
} }
// Store sha256 // Store sha256
let hash = hex::encode(sha2::Sha256::digest(&tarball_bytes)); let hash = format!("{:x}", sha2::Sha256::digest(&tarball_bytes));
let hash_key = format!("{}.sha256", tarball_key); let hash_key = format!("{}.sha256", tarball_key);
let _ = state.storage.put(&hash_key, hash.as_bytes()).await; let _ = state.storage.put(&hash_key, hash.as_bytes()).await;
} }
// Merge versions // Merge versions
let Some(meta_obj) = metadata.as_object_mut() else { let meta_obj = metadata.as_object_mut().unwrap();
return (StatusCode::INTERNAL_SERVER_ERROR, "invalid metadata format").into_response();
};
let stored_versions = meta_obj.entry("versions").or_insert(serde_json::json!({})); let stored_versions = meta_obj.entry("versions").or_insert(serde_json::json!({}));
if let Some(sv) = stored_versions.as_object_mut() { if let Some(sv) = stored_versions.as_object_mut() {
for (ver, ver_data) in new_versions { for (ver, ver_data) in new_versions {
@@ -418,6 +419,25 @@ async fn handle_publish(
// Helpers // Helpers
// ============================================================================ // ============================================================================
async fn fetch_from_proxy(
client: &reqwest::Client,
url: &str,
timeout_secs: u64,
auth: Option<&str>,
) -> Result<Vec<u8>, ()> {
let mut request = client.get(url).timeout(Duration::from_secs(timeout_secs));
if let Some(credentials) = auth {
request = request.header("Authorization", basic_auth_header(credentials));
}
let response = request.send().await.map_err(|_| ())?;
if !response.status().is_success() {
return Err(());
}
response.bytes().await.map(|b| b.to_vec()).map_err(|_| ())
}
fn with_content_type( fn with_content_type(
is_tarball: bool, is_tarball: bool,
data: Bytes, data: Bytes,
@@ -432,7 +452,6 @@ fn with_content_type(
} }
#[cfg(test)] #[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests { mod tests {
use super::*; use super::*;
@@ -556,229 +575,4 @@ mod tests {
assert!(!is_valid_attachment_name("")); assert!(!is_valid_attachment_name(""));
assert!(!is_valid_attachment_name("foo\0bar.tgz")); assert!(!is_valid_attachment_name("foo\0bar.tgz"));
} }
#[test]
fn test_with_content_type_tarball() {
let data = Bytes::from("tarball-data");
let (status, headers, body) = with_content_type(true, data.clone());
assert_eq!(status, StatusCode::OK);
assert_eq!(headers[0].1, "application/octet-stream");
assert_eq!(body, data);
}
#[test]
fn test_with_content_type_json() {
let data = Bytes::from("json-data");
let (status, headers, body) = with_content_type(false, data.clone());
assert_eq!(status, StatusCode::OK);
assert_eq!(headers[0].1, "application/json");
assert_eq!(body, data);
}
#[test]
fn test_rewrite_tarball_urls_trailing_slash() {
let metadata = serde_json::json!({
"name": "test",
"versions": {
"1.0.0": {
"dist": {
"tarball": "https://registry.npmjs.org/test/-/test-1.0.0.tgz"
}
}
}
});
let data = serde_json::to_vec(&metadata).unwrap();
let result =
rewrite_tarball_urls(&data, "http://nora:5000/", "https://registry.npmjs.org/")
.unwrap();
let json: serde_json::Value = serde_json::from_slice(&result).unwrap();
let tarball = json["versions"]["1.0.0"]["dist"]["tarball"]
.as_str()
.unwrap();
assert!(tarball.starts_with("http://nora:5000/npm/"));
}
#[test]
fn test_rewrite_tarball_urls_preserves_other_fields() {
let metadata = serde_json::json!({
"name": "test",
"description": "A test package",
"versions": {
"1.0.0": {
"dist": {
"tarball": "https://registry.npmjs.org/test/-/test-1.0.0.tgz",
"shasum": "abc123"
},
"dependencies": {"lodash": "^4.0.0"}
}
}
});
let data = serde_json::to_vec(&metadata).unwrap();
let result =
rewrite_tarball_urls(&data, "http://nora:5000", "https://registry.npmjs.org").unwrap();
let json: serde_json::Value = serde_json::from_slice(&result).unwrap();
assert_eq!(json["description"], "A test package");
assert_eq!(json["versions"]["1.0.0"]["dist"]["shasum"], "abc123");
}
#[test]
fn test_is_valid_attachment_name_valid() {
assert!(is_valid_attachment_name("package-1.0.0.tgz"));
assert!(is_valid_attachment_name("@scope-pkg-2.0.tgz"));
assert!(is_valid_attachment_name("my_pkg.tgz"));
}
#[test]
fn test_is_valid_attachment_name_traversal() {
assert!(!is_valid_attachment_name("../etc/passwd"));
assert!(!is_valid_attachment_name("foo/../bar"));
}
#[test]
fn test_is_valid_attachment_name_slash() {
assert!(!is_valid_attachment_name("path/file.tgz"));
assert!(!is_valid_attachment_name("path\\file.tgz"));
}
#[test]
fn test_is_valid_attachment_name_null_byte() {
assert!(!is_valid_attachment_name("file\0.tgz"));
}
#[test]
fn test_is_valid_attachment_name_empty() {
assert!(!is_valid_attachment_name(""));
}
#[test]
fn test_is_valid_attachment_name_special_chars() {
assert!(!is_valid_attachment_name("file name.tgz")); // space
assert!(!is_valid_attachment_name("file;cmd.tgz")); // semicolon
}
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod integration_tests {
use crate::test_helpers::{body_bytes, create_test_context, send};
use axum::body::Body;
use axum::http::{Method, StatusCode};
use base64::Engine;
#[tokio::test]
async fn test_npm_metadata_from_cache() {
let ctx = create_test_context();
let metadata = serde_json::json!({
"name": "lodash",
"versions": {
"4.17.21": { "dist": { "tarball": "http://example.com/lodash.tgz" } }
}
});
let metadata_bytes = serde_json::to_vec(&metadata).unwrap();
ctx.state
.storage
.put("npm/lodash/metadata.json", &metadata_bytes)
.await
.unwrap();
let response = send(&ctx.app, Method::GET, "/npm/lodash", "").await;
assert_eq!(response.status(), StatusCode::OK);
let body = body_bytes(response).await;
let json: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert_eq!(json["name"], "lodash");
}
#[tokio::test]
async fn test_npm_tarball_from_cache() {
let ctx = create_test_context();
let tarball_data = b"fake-tarball-bytes";
ctx.state
.storage
.put("npm/lodash/tarballs/lodash-4.17.21.tgz", tarball_data)
.await
.unwrap();
let response = send(
&ctx.app,
Method::GET,
"/npm/lodash/-/lodash-4.17.21.tgz",
"",
)
.await;
assert_eq!(response.status(), StatusCode::OK);
let body = body_bytes(response).await;
assert_eq!(&body[..], tarball_data);
}
#[tokio::test]
async fn test_npm_not_found_no_proxy() {
let ctx = create_test_context();
// No proxy configured, no local data
let response = send(&ctx.app, Method::GET, "/npm/nonexistent", "").await;
assert_eq!(response.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn test_npm_publish_basic() {
let ctx = create_test_context();
let tarball_data = b"fake-tarball";
let base64_data = base64::engine::general_purpose::STANDARD.encode(tarball_data);
let payload = serde_json::json!({
"name": "mypkg",
"versions": {
"1.0.0": { "dist": {} }
},
"_attachments": {
"mypkg-1.0.0.tgz": { "data": base64_data }
},
"dist-tags": { "latest": "1.0.0" }
});
let body_bytes = serde_json::to_vec(&payload).unwrap();
let response = send(&ctx.app, Method::PUT, "/npm/mypkg", Body::from(body_bytes)).await;
assert_eq!(response.status(), StatusCode::CREATED);
// Verify tarball was stored
let stored_tarball = ctx
.state
.storage
.get("npm/mypkg/tarballs/mypkg-1.0.0.tgz")
.await
.unwrap();
assert_eq!(&stored_tarball[..], tarball_data);
}
#[tokio::test]
async fn test_npm_publish_name_mismatch() {
let ctx = create_test_context();
let tarball_data = b"fake-tarball";
let base64_data = base64::engine::general_purpose::STANDARD.encode(tarball_data);
let payload = serde_json::json!({
"name": "other",
"versions": {
"1.0.0": { "dist": {} }
},
"_attachments": {
"other-1.0.0.tgz": { "data": base64_data }
},
"dist-tags": { "latest": "1.0.0" }
});
let body_bytes = serde_json::to_vec(&payload).unwrap();
let response = send(&ctx.app, Method::PUT, "/npm/mypkg", Body::from(body_bytes)).await;
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
}
} }

File diff suppressed because it is too large Load Diff

View File

@@ -141,191 +141,3 @@ fn guess_content_type(path: &str) -> &'static str {
_ => "application/octet-stream", _ => "application/octet-stream",
} }
} }
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_guess_content_type_json() {
assert_eq!(guess_content_type("config.json"), "application/json");
}
#[test]
fn test_guess_content_type_xml() {
assert_eq!(guess_content_type("data.xml"), "application/xml");
}
#[test]
fn test_guess_content_type_html() {
assert_eq!(guess_content_type("index.html"), "text/html");
assert_eq!(guess_content_type("page.htm"), "text/html");
}
#[test]
fn test_guess_content_type_css() {
assert_eq!(guess_content_type("style.css"), "text/css");
}
#[test]
fn test_guess_content_type_js() {
assert_eq!(guess_content_type("app.js"), "application/javascript");
}
#[test]
fn test_guess_content_type_text() {
assert_eq!(guess_content_type("readme.txt"), "text/plain");
}
#[test]
fn test_guess_content_type_markdown() {
assert_eq!(guess_content_type("README.md"), "text/markdown");
}
#[test]
fn test_guess_content_type_yaml() {
assert_eq!(guess_content_type("config.yaml"), "application/x-yaml");
assert_eq!(guess_content_type("config.yml"), "application/x-yaml");
}
#[test]
fn test_guess_content_type_toml() {
assert_eq!(guess_content_type("Cargo.toml"), "application/toml");
}
#[test]
fn test_guess_content_type_archives() {
assert_eq!(guess_content_type("data.tar"), "application/x-tar");
assert_eq!(guess_content_type("data.gz"), "application/gzip");
assert_eq!(guess_content_type("data.gzip"), "application/gzip");
assert_eq!(guess_content_type("data.zip"), "application/zip");
}
#[test]
fn test_guess_content_type_images() {
assert_eq!(guess_content_type("logo.png"), "image/png");
assert_eq!(guess_content_type("photo.jpg"), "image/jpeg");
assert_eq!(guess_content_type("photo.jpeg"), "image/jpeg");
assert_eq!(guess_content_type("anim.gif"), "image/gif");
assert_eq!(guess_content_type("icon.svg"), "image/svg+xml");
}
#[test]
fn test_guess_content_type_special() {
assert_eq!(guess_content_type("doc.pdf"), "application/pdf");
assert_eq!(guess_content_type("module.wasm"), "application/wasm");
}
#[test]
fn test_guess_content_type_unknown() {
assert_eq!(guess_content_type("binary.bin"), "application/octet-stream");
assert_eq!(guess_content_type("noext"), "application/octet-stream");
}
#[test]
fn test_guess_content_type_case_insensitive() {
assert_eq!(guess_content_type("FILE.JSON"), "application/json");
assert_eq!(guess_content_type("IMAGE.PNG"), "image/png");
}
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod integration_tests {
use crate::storage::{Storage, StorageError};
use crate::test_helpers::{
body_bytes, create_test_context, create_test_context_with_raw_disabled, send,
};
use axum::http::{Method, StatusCode};
#[tokio::test]
async fn test_raw_put_get_roundtrip() {
let ctx = create_test_context();
let put_resp = send(&ctx.app, Method::PUT, "/raw/test.txt", b"hello".to_vec()).await;
assert_eq!(put_resp.status(), StatusCode::CREATED);
let get_resp = send(&ctx.app, Method::GET, "/raw/test.txt", "").await;
assert_eq!(get_resp.status(), StatusCode::OK);
let body = body_bytes(get_resp).await;
assert_eq!(&body[..], b"hello");
}
#[tokio::test]
async fn test_raw_head() {
let ctx = create_test_context();
send(
&ctx.app,
Method::PUT,
"/raw/test.txt",
b"hello world".to_vec(),
)
.await;
let head_resp = send(&ctx.app, Method::HEAD, "/raw/test.txt", "").await;
assert_eq!(head_resp.status(), StatusCode::OK);
let cl = head_resp.headers().get("content-length").unwrap();
assert_eq!(cl.to_str().unwrap(), "11");
}
#[tokio::test]
async fn test_raw_delete() {
let ctx = create_test_context();
send(&ctx.app, Method::PUT, "/raw/test.txt", b"data".to_vec()).await;
let del = send(&ctx.app, Method::DELETE, "/raw/test.txt", "").await;
assert_eq!(del.status(), StatusCode::NO_CONTENT);
let get = send(&ctx.app, Method::GET, "/raw/test.txt", "").await;
assert_eq!(get.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn test_raw_not_found() {
let ctx = create_test_context();
let resp = send(&ctx.app, Method::GET, "/raw/missing.txt", "").await;
assert_eq!(resp.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn test_raw_content_type_json() {
let ctx = create_test_context();
send(&ctx.app, Method::PUT, "/raw/file.json", b"{}".to_vec()).await;
let resp = send(&ctx.app, Method::GET, "/raw/file.json", "").await;
assert_eq!(resp.status(), StatusCode::OK);
let ct = resp.headers().get("content-type").unwrap();
assert_eq!(ct.to_str().unwrap(), "application/json");
}
#[tokio::test]
async fn test_raw_payload_too_large() {
let ctx = create_test_context();
let big = vec![0u8; 2 * 1024 * 1024]; // 2 MB > 1 MB limit
let resp = send(&ctx.app, Method::PUT, "/raw/large.bin", big).await;
assert_eq!(resp.status(), StatusCode::PAYLOAD_TOO_LARGE);
}
#[tokio::test]
async fn test_raw_disabled() {
let ctx = create_test_context_with_raw_disabled();
let get = send(&ctx.app, Method::GET, "/raw/test.txt", "").await;
assert_eq!(get.status(), StatusCode::NOT_FOUND);
let put = send(&ctx.app, Method::PUT, "/raw/test.txt", b"data".to_vec()).await;
assert_eq!(put.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn test_upload_path_traversal_rejected() {
let temp_dir = tempfile::TempDir::new().unwrap();
let storage = Storage::new_local(temp_dir.path().to_str().unwrap());
let result = storage.put("raw/../../../etc/passwd", b"pwned").await;
assert!(result.is_err(), "path traversal key must be rejected");
match result {
Err(StorageError::Validation(v)) => {
assert_eq!(format!("{}", v), "Path traversal detected");
}
other => panic!("expected Validation(PathTraversal), got {:?}", other),
}
}
}

View File

@@ -3,7 +3,7 @@
//! In-memory repository index with lazy rebuild on invalidation. //! In-memory repository index with lazy rebuild on invalidation.
//! //!
//! Design: //! Design (designed for efficiency):
//! - Rebuild happens ONLY on write operations, not TTL //! - Rebuild happens ONLY on write operations, not TTL
//! - Double-checked locking prevents duplicate rebuilds //! - Double-checked locking prevents duplicate rebuilds
//! - Arc<Vec> for zero-cost reads //! - Arc<Vec> for zero-cost reads
@@ -80,8 +80,6 @@ pub struct RepoIndex {
pub npm: RegistryIndex, pub npm: RegistryIndex,
pub cargo: RegistryIndex, pub cargo: RegistryIndex,
pub pypi: RegistryIndex, pub pypi: RegistryIndex,
pub go: RegistryIndex,
pub raw: RegistryIndex,
} }
impl RepoIndex { impl RepoIndex {
@@ -92,8 +90,6 @@ impl RepoIndex {
npm: RegistryIndex::new(), npm: RegistryIndex::new(),
cargo: RegistryIndex::new(), cargo: RegistryIndex::new(),
pypi: RegistryIndex::new(), pypi: RegistryIndex::new(),
go: RegistryIndex::new(),
raw: RegistryIndex::new(),
} }
} }
@@ -105,8 +101,6 @@ impl RepoIndex {
"npm" => self.npm.invalidate(), "npm" => self.npm.invalidate(),
"cargo" => self.cargo.invalidate(), "cargo" => self.cargo.invalidate(),
"pypi" => self.pypi.invalidate(), "pypi" => self.pypi.invalidate(),
"go" => self.go.invalidate(),
"raw" => self.raw.invalidate(),
_ => {} _ => {}
} }
} }
@@ -119,8 +113,6 @@ impl RepoIndex {
"npm" => &self.npm, "npm" => &self.npm,
"cargo" => &self.cargo, "cargo" => &self.cargo,
"pypi" => &self.pypi, "pypi" => &self.pypi,
"go" => &self.go,
"raw" => &self.raw,
_ => return Arc::new(Vec::new()), _ => return Arc::new(Vec::new()),
}; };
@@ -140,8 +132,6 @@ impl RepoIndex {
"npm" => build_npm_index(storage).await, "npm" => build_npm_index(storage).await,
"cargo" => build_cargo_index(storage).await, "cargo" => build_cargo_index(storage).await,
"pypi" => build_pypi_index(storage).await, "pypi" => build_pypi_index(storage).await,
"go" => build_go_index(storage).await,
"raw" => build_raw_index(storage).await,
_ => Vec::new(), _ => Vec::new(),
}; };
info!(registry = registry, count = data.len(), "Index rebuilt"); info!(registry = registry, count = data.len(), "Index rebuilt");
@@ -152,15 +142,13 @@ impl RepoIndex {
} }
/// Get counts for stats (no rebuild, just current state) /// Get counts for stats (no rebuild, just current state)
pub fn counts(&self) -> (usize, usize, usize, usize, usize, usize, usize) { pub fn counts(&self) -> (usize, usize, usize, usize, usize) {
( (
self.docker.count(), self.docker.count(),
self.maven.count(), self.maven.count(),
self.npm.count(), self.npm.count(),
self.cargo.count(), self.cargo.count(),
self.pypi.count(), self.pypi.count(),
self.go.count(),
self.raw.count(),
) )
} }
} }
@@ -185,41 +173,35 @@ async fn build_docker_index(storage: &Storage) -> Vec<RepoInfo> {
} }
if let Some(rest) = key.strip_prefix("docker/") { if let Some(rest) = key.strip_prefix("docker/") {
// Support both single-segment and namespaced images:
// docker/alpine/manifests/latest.json → name="alpine"
// docker/library/alpine/manifests/latest.json → name="library/alpine"
let parts: Vec<_> = rest.split('/').collect(); let parts: Vec<_> = rest.split('/').collect();
let manifest_pos = parts.iter().position(|&p| p == "manifests"); if parts.len() >= 3 && parts[1] == "manifests" && key.ends_with(".json") {
if let Some(pos) = manifest_pos { let name = parts[0].to_string();
if pos >= 1 && key.ends_with(".json") { let entry = repos.entry(name).or_insert((0, 0, 0));
let name = parts[..pos].join("/"); entry.0 += 1;
let entry = repos.entry(name).or_insert((0, 0, 0));
entry.0 += 1;
if let Ok(data) = storage.get(key).await { if let Ok(data) = storage.get(key).await {
if let Ok(m) = serde_json::from_slice::<serde_json::Value>(&data) { if let Ok(m) = serde_json::from_slice::<serde_json::Value>(&data) {
let cfg = m let cfg = m
.get("config") .get("config")
.and_then(|c| c.get("size")) .and_then(|c| c.get("size"))
.and_then(|s| s.as_u64()) .and_then(|s| s.as_u64())
.unwrap_or(0); .unwrap_or(0);
let layers: u64 = m let layers: u64 = m
.get("layers") .get("layers")
.and_then(|l| l.as_array()) .and_then(|l| l.as_array())
.map(|arr| { .map(|arr| {
arr.iter() arr.iter()
.filter_map(|l| l.get("size").and_then(|s| s.as_u64())) .filter_map(|l| l.get("size").and_then(|s| s.as_u64()))
.sum() .sum()
}) })
.unwrap_or(0); .unwrap_or(0);
entry.1 += cfg + layers; entry.1 += cfg + layers;
}
} }
}
if let Some(meta) = storage.stat(key).await { if let Some(meta) = storage.stat(key).await {
if meta.modified > entry.2 { if meta.modified > entry.2 {
entry.2 = meta.modified; entry.2 = meta.modified;
}
} }
} }
} }
@@ -341,57 +323,6 @@ async fn build_pypi_index(storage: &Storage) -> Vec<RepoInfo> {
to_sorted_vec(packages) to_sorted_vec(packages)
} }
async fn build_go_index(storage: &Storage) -> Vec<RepoInfo> {
let keys = storage.list("go/").await;
let mut modules: HashMap<String, (usize, u64, u64)> = HashMap::new();
for key in &keys {
if let Some(rest) = key.strip_prefix("go/") {
// Pattern: go/{module}/@v/{version}.zip
// Count .zip files as versions (authoritative artifacts)
if rest.contains("/@v/") && key.ends_with(".zip") {
// Extract module path: everything before /@v/
if let Some(pos) = rest.rfind("/@v/") {
let module = &rest[..pos];
let entry = modules.entry(module.to_string()).or_insert((0, 0, 0));
entry.0 += 1;
if let Some(meta) = storage.stat(key).await {
entry.1 += meta.size;
if meta.modified > entry.2 {
entry.2 = meta.modified;
}
}
}
}
}
}
to_sorted_vec(modules)
}
async fn build_raw_index(storage: &Storage) -> Vec<RepoInfo> {
let keys = storage.list("raw/").await;
let mut files: HashMap<String, (usize, u64, u64)> = HashMap::new();
for key in &keys {
if let Some(rest) = key.strip_prefix("raw/") {
// Group by top-level directory
let group = rest.split('/').next().unwrap_or(rest).to_string();
let entry = files.entry(group).or_insert((0, 0, 0));
entry.0 += 1;
if let Some(meta) = storage.stat(key).await {
entry.1 += meta.size;
if meta.modified > entry.2 {
entry.2 = meta.modified;
}
}
}
}
to_sorted_vec(files)
}
/// Convert HashMap to sorted Vec<RepoInfo> /// Convert HashMap to sorted Vec<RepoInfo>
fn to_sorted_vec(map: HashMap<String, (usize, u64, u64)>) -> Vec<RepoInfo> { fn to_sorted_vec(map: HashMap<String, (usize, u64, u64)>) -> Vec<RepoInfo> {
let mut result: Vec<_> = map let mut result: Vec<_> = map
@@ -424,165 +355,3 @@ pub fn paginate<T: Clone>(data: &[T], page: usize, limit: usize) -> (Vec<T>, usi
let end = (start + limit).min(total); let end = (start + limit).min(total);
(data[start..end].to_vec(), total) (data[start..end].to_vec(), total)
} }
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_paginate_first_page() {
let data = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
let (page, total) = paginate(&data, 1, 3);
assert_eq!(page, vec![1, 2, 3]);
assert_eq!(total, 10);
}
#[test]
fn test_paginate_second_page() {
let data = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
let (page, total) = paginate(&data, 2, 3);
assert_eq!(page, vec![4, 5, 6]);
assert_eq!(total, 10);
}
#[test]
fn test_paginate_last_page_partial() {
let data = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
let (page, total) = paginate(&data, 4, 3);
assert_eq!(page, vec![10]);
assert_eq!(total, 10);
}
#[test]
fn test_paginate_beyond_range() {
let data = vec![1, 2, 3];
let (page, total) = paginate(&data, 5, 3);
assert!(page.is_empty());
assert_eq!(total, 3);
}
#[test]
fn test_paginate_empty_data() {
let data: Vec<i32> = vec![];
let (page, total) = paginate(&data, 1, 10);
assert!(page.is_empty());
assert_eq!(total, 0);
}
#[test]
fn test_paginate_page_zero() {
// page 0 with saturating_sub becomes 0, so start = 0
let data = vec![1, 2, 3];
let (page, _) = paginate(&data, 0, 2);
assert_eq!(page, vec![1, 2]);
}
#[test]
fn test_paginate_large_limit() {
let data = vec![1, 2, 3];
let (page, total) = paginate(&data, 1, 100);
assert_eq!(page, vec![1, 2, 3]);
assert_eq!(total, 3);
}
#[test]
fn test_registry_index_new() {
let idx = RegistryIndex::new();
assert_eq!(idx.count(), 0);
assert!(idx.is_dirty());
}
#[test]
fn test_registry_index_invalidate() {
let idx = RegistryIndex::new();
// Initially dirty
assert!(idx.is_dirty());
// Set data clears dirty
idx.set(vec![RepoInfo {
name: "test".to_string(),
versions: 1,
size: 100,
updated: "2026-01-01".to_string(),
}]);
assert!(!idx.is_dirty());
assert_eq!(idx.count(), 1);
// Invalidate makes it dirty again
idx.invalidate();
assert!(idx.is_dirty());
}
#[test]
fn test_registry_index_get_cached() {
let idx = RegistryIndex::new();
idx.set(vec![
RepoInfo {
name: "a".to_string(),
versions: 2,
size: 200,
updated: "today".to_string(),
},
RepoInfo {
name: "b".to_string(),
versions: 1,
size: 100,
updated: "yesterday".to_string(),
},
]);
let cached = idx.get_cached();
assert_eq!(cached.len(), 2);
assert_eq!(cached[0].name, "a");
}
#[test]
fn test_registry_index_default() {
let idx = RegistryIndex::default();
assert_eq!(idx.count(), 0);
}
#[test]
fn test_repo_index_new() {
let idx = RepoIndex::new();
let (d, m, n, c, p, g, r) = idx.counts();
assert_eq!((d, m, n, c, p, g, r), (0, 0, 0, 0, 0, 0, 0));
}
#[test]
fn test_repo_index_invalidate() {
let idx = RepoIndex::new();
// Should not panic for any registry
idx.invalidate("docker");
idx.invalidate("maven");
idx.invalidate("npm");
idx.invalidate("cargo");
idx.invalidate("pypi");
idx.invalidate("raw");
idx.invalidate("unknown"); // should be a no-op
}
#[test]
fn test_repo_index_default() {
let idx = RepoIndex::default();
let (d, m, n, c, p, g, r) = idx.counts();
assert_eq!((d, m, n, c, p, g, r), (0, 0, 0, 0, 0, 0, 0));
}
#[test]
fn test_to_sorted_vec() {
let mut map = std::collections::HashMap::new();
map.insert("zebra".to_string(), (3usize, 100u64, 0u64));
map.insert("alpha".to_string(), (1, 50, 1700000000));
let result = to_sorted_vec(map);
assert_eq!(result.len(), 2);
assert_eq!(result[0].name, "alpha");
assert_eq!(result[0].versions, 1);
assert_eq!(result[0].size, 50);
assert_ne!(result[0].updated, "N/A");
assert_eq!(result[1].name, "zebra");
assert_eq!(result[1].versions, 3);
assert_eq!(result[1].updated, "N/A"); // modified = 0
}
}

View File

@@ -92,69 +92,4 @@ mod tests {
let cloned = id.clone(); let cloned = id.clone();
assert_eq!(id.0, cloned.0); assert_eq!(id.0, cloned.0);
} }
#[test]
fn test_request_id_debug() {
let id = RequestId("abc-def".to_string());
let debug = format!("{:?}", id);
assert!(debug.contains("abc-def"));
}
#[test]
fn test_request_id_header_name() {
assert_eq!(REQUEST_ID_HEADER.as_str(), "x-request-id");
}
#[test]
fn test_request_id_deref_string_methods() {
let id = RequestId("req-12345".to_string());
assert!(id.starts_with("req-"));
assert_eq!(id.len(), 9);
}
}
#[cfg(test)]
#[allow(clippy::unwrap_used)]
mod integration_tests {
use crate::test_helpers::{create_test_context, send, send_with_headers};
use axum::http::{Method, StatusCode};
#[tokio::test]
async fn test_response_has_request_id() {
let ctx = create_test_context();
let response = send(&ctx.app, Method::GET, "/health", "").await;
assert_eq!(response.status(), StatusCode::OK);
let request_id = response.headers().get("x-request-id");
assert!(
request_id.is_some(),
"Response must have X-Request-ID header"
);
let value = request_id.unwrap().to_str().unwrap();
assert!(!value.is_empty(), "X-Request-ID must not be empty");
}
#[tokio::test]
async fn test_preserves_incoming_request_id() {
let ctx = create_test_context();
let custom_id = "custom-123";
let response = send_with_headers(
&ctx.app,
Method::GET,
"/health",
vec![("x-request-id", custom_id)],
"",
)
.await;
assert_eq!(response.status(), StatusCode::OK);
let returned_id = response
.headers()
.get("x-request-id")
.unwrap()
.to_str()
.unwrap();
assert_eq!(returned_id, custom_id);
}
} }

View File

@@ -72,7 +72,6 @@ impl SecretsProvider for EnvProvider {
} }
#[cfg(test)] #[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests { mod tests {
use super::*; use super::*;

View File

@@ -130,7 +130,6 @@ pub fn create_secrets_provider(
} }
#[cfg(test)] #[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests { mod tests {
use super::*; use super::*;

View File

@@ -33,8 +33,8 @@ impl ProtectedString {
} }
/// Consume and return the inner value /// Consume and return the inner value
pub fn into_inner(mut self) -> Zeroizing<String> { pub fn into_inner(self) -> Zeroizing<String> {
Zeroizing::new(std::mem::take(&mut self.inner)) Zeroizing::new(self.inner.clone())
} }
/// Check if the secret is empty /// Check if the secret is empty
@@ -74,8 +74,8 @@ impl From<&str> for ProtectedString {
#[derive(Clone, Zeroize)] #[derive(Clone, Zeroize)]
#[zeroize(drop)] #[zeroize(drop)]
pub struct S3Credentials { pub struct S3Credentials {
#[zeroize(skip)] // access_key_id is not sensitive
pub access_key_id: String, pub access_key_id: String,
#[zeroize(skip)] // access_key_id is not sensitive
pub secret_access_key: ProtectedString, pub secret_access_key: ProtectedString,
pub region: Option<String>, pub region: Option<String>,
} }

View File

@@ -68,6 +68,10 @@ impl StorageBackend for LocalStorage {
async fn get(&self, key: &str) -> Result<Bytes> { async fn get(&self, key: &str) -> Result<Bytes> {
let path = self.key_to_path(key); let path = self.key_to_path(key);
if !path.exists() {
return Err(StorageError::NotFound);
}
let mut file = fs::File::open(&path).await.map_err(|e| { let mut file = fs::File::open(&path).await.map_err(|e| {
if e.kind() == std::io::ErrorKind::NotFound { if e.kind() == std::io::ErrorKind::NotFound {
StorageError::NotFound StorageError::NotFound
@@ -87,13 +91,13 @@ impl StorageBackend for LocalStorage {
async fn delete(&self, key: &str) -> Result<()> { async fn delete(&self, key: &str) -> Result<()> {
let path = self.key_to_path(key); let path = self.key_to_path(key);
fs::remove_file(&path).await.map_err(|e| { if !path.exists() {
if e.kind() == std::io::ErrorKind::NotFound { return Err(StorageError::NotFound);
StorageError::NotFound }
} else {
StorageError::Io(e.to_string()) fs::remove_file(&path)
} .await
})?; .map_err(|e| StorageError::Io(e.to_string()))?;
Ok(()) Ok(())
} }
@@ -138,36 +142,12 @@ impl StorageBackend for LocalStorage {
fs::create_dir_all(&self.base_path).await.is_ok() fs::create_dir_all(&self.base_path).await.is_ok()
} }
async fn total_size(&self) -> u64 {
let base = self.base_path.clone();
tokio::task::spawn_blocking(move || {
fn dir_size(path: &std::path::Path) -> u64 {
let mut total = 0u64;
if let Ok(entries) = std::fs::read_dir(path) {
for entry in entries.flatten() {
let path = entry.path();
if path.is_file() {
total += entry.metadata().map(|m| m.len()).unwrap_or(0);
} else if path.is_dir() {
total += dir_size(&path);
}
}
}
total
}
dir_size(&base)
})
.await
.unwrap_or(0)
}
fn backend_name(&self) -> &'static str { fn backend_name(&self) -> &'static str {
"local" "local"
} }
} }
#[cfg(test)] #[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests { mod tests {
use super::*; use super::*;
use tempfile::TempDir; use tempfile::TempDir;
@@ -274,147 +254,4 @@ mod tests {
let storage = LocalStorage::new(temp_dir.path().to_str().unwrap()); let storage = LocalStorage::new(temp_dir.path().to_str().unwrap());
assert_eq!(storage.backend_name(), "local"); assert_eq!(storage.backend_name(), "local");
} }
#[tokio::test(flavor = "multi_thread")]
async fn test_concurrent_writes_same_key() {
let temp_dir = TempDir::new().unwrap();
let storage = std::sync::Arc::new(LocalStorage::new(temp_dir.path().to_str().unwrap()));
let mut handles = Vec::new();
for i in 0..10u8 {
let s = storage.clone();
handles.push(tokio::spawn(async move {
let data = vec![i; 1024];
s.put("shared/key", &data).await
}));
}
for h in handles {
h.await.expect("task panicked").expect("put failed");
}
let data = storage.get("shared/key").await.expect("get failed");
assert_eq!(data.len(), 1024);
let first = data[0];
assert!(
data.iter().all(|&b| b == first),
"file is corrupted — mixed writers"
);
}
#[tokio::test(flavor = "multi_thread")]
async fn test_concurrent_writes_different_keys() {
let temp_dir = TempDir::new().unwrap();
let storage = std::sync::Arc::new(LocalStorage::new(temp_dir.path().to_str().unwrap()));
let mut handles = Vec::new();
for i in 0..10u32 {
let s = storage.clone();
handles.push(tokio::spawn(async move {
let key = format!("key/{}", i);
s.put(&key, format!("data-{}", i).as_bytes()).await
}));
}
for h in handles {
h.await.expect("task panicked").expect("put failed");
}
for i in 0..10u32 {
let key = format!("key/{}", i);
let data = storage.get(&key).await.expect("get failed");
assert_eq!(&*data, format!("data-{}", i).as_bytes());
}
}
#[tokio::test(flavor = "multi_thread")]
async fn test_concurrent_read_during_write() {
let temp_dir = TempDir::new().unwrap();
let storage = std::sync::Arc::new(LocalStorage::new(temp_dir.path().to_str().unwrap()));
let old_data = vec![0u8; 4096];
storage.put("rw/key", &old_data).await.expect("seed put");
let new_data = vec![1u8; 4096];
let sw = storage.clone();
let writer = tokio::spawn(async move {
sw.put("rw/key", &new_data).await.expect("put failed");
});
let sr = storage.clone();
let reader = tokio::spawn(async move {
match sr.get("rw/key").await {
Ok(_data) => {
// tokio::fs::write is not atomic, so partial reads
// (mix of old and new bytes) are expected — not a bug.
// We only verify the final state after both tasks complete.
}
Err(crate::storage::StorageError::NotFound) => {}
Err(e) => panic!("unexpected error: {}", e),
}
});
writer.await.expect("writer panicked");
reader.await.expect("reader panicked");
let data = storage.get("rw/key").await.expect("final get");
assert_eq!(&*data, &vec![1u8; 4096]);
}
#[tokio::test]
async fn test_total_size_empty() {
let temp_dir = TempDir::new().unwrap();
let storage = LocalStorage::new(temp_dir.path().to_str().unwrap());
assert_eq!(storage.total_size().await, 0);
}
#[tokio::test]
async fn test_total_size_with_files() {
let temp_dir = TempDir::new().unwrap();
let storage = LocalStorage::new(temp_dir.path().to_str().unwrap());
storage.put("a/file1", b"hello").await.unwrap(); // 5 bytes
storage.put("b/file2", b"world!").await.unwrap(); // 6 bytes
let size = storage.total_size().await;
assert_eq!(size, 11);
}
#[tokio::test]
async fn test_total_size_after_delete() {
let temp_dir = TempDir::new().unwrap();
let storage = LocalStorage::new(temp_dir.path().to_str().unwrap());
storage.put("file1", b"12345").await.unwrap();
storage.put("file2", b"67890").await.unwrap();
assert_eq!(storage.total_size().await, 10);
storage.delete("file1").await.unwrap();
assert_eq!(storage.total_size().await, 5);
}
#[tokio::test(flavor = "multi_thread")]
async fn test_concurrent_deletes_same_key() {
let temp_dir = TempDir::new().unwrap();
let storage = std::sync::Arc::new(LocalStorage::new(temp_dir.path().to_str().unwrap()));
storage.put("del/key", b"ephemeral").await.expect("put");
let mut handles = Vec::new();
for _ in 0..10 {
let s = storage.clone();
handles.push(tokio::spawn(async move {
let _ = s.delete("del/key").await;
}));
}
for h in handles {
h.await.expect("task panicked");
}
assert!(matches!(
storage.get("del/key").await,
Err(crate::storage::StorageError::NotFound)
));
}
} }

View File

@@ -46,8 +46,6 @@ pub trait StorageBackend: Send + Sync {
async fn list(&self, prefix: &str) -> Vec<String>; async fn list(&self, prefix: &str) -> Vec<String>;
async fn stat(&self, key: &str) -> Option<FileMeta>; async fn stat(&self, key: &str) -> Option<FileMeta>;
async fn health_check(&self) -> bool; async fn health_check(&self) -> bool;
/// Total size of all stored artifacts in bytes
async fn total_size(&self) -> u64;
fn backend_name(&self) -> &'static str; fn backend_name(&self) -> &'static str;
} }
@@ -112,10 +110,6 @@ impl Storage {
self.inner.health_check().await self.inner.health_check().await
} }
pub async fn total_size(&self) -> u64 {
self.inner.total_size().await
}
pub fn backend_name(&self) -> &'static str { pub fn backend_name(&self) -> &'static str {
self.inner.backend_name() self.inner.backend_name()
} }

View File

@@ -4,7 +4,7 @@
use async_trait::async_trait; use async_trait::async_trait;
use axum::body::Bytes; use axum::body::Bytes;
use chrono::Utc; use chrono::Utc;
use hmac::{digest::KeyInit, Hmac, Mac}; use hmac::{Hmac, Mac};
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use super::{FileMeta, Result, StorageBackend, StorageError}; use super::{FileMeta, Result, StorageBackend, StorageError};
@@ -79,8 +79,7 @@ impl S3Storage {
method, canonical_uri, canonical_query, canonical_headers, signed_headers, payload_hash method, canonical_uri, canonical_query, canonical_headers, signed_headers, payload_hash
); );
let canonical_request_hash = let canonical_request_hash = hex::encode(Sha256::digest(canonical_request.as_bytes()));
hex::encode(sha2::Sha256::digest(canonical_request.as_bytes()));
// String to sign // String to sign
let credential_scope = format!("{}/{}/s3/aws4_request", date, self.region); let credential_scope = format!("{}/{}/s3/aws4_request", date, self.region);
@@ -258,8 +257,7 @@ impl StorageBackend for S3Storage {
canonical_uri, canonical_headers, signed_headers, payload_hash canonical_uri, canonical_headers, signed_headers, payload_hash
); );
let canonical_request_hash = let canonical_request_hash = hex::encode(Sha256::digest(canonical_request.as_bytes()));
hex::encode(sha2::Sha256::digest(canonical_request.as_bytes()));
let credential_scope = format!("{}/{}/s3/aws4_request", date, self.region); let credential_scope = format!("{}/{}/s3/aws4_request", date, self.region);
let string_to_sign = format!( let string_to_sign = format!(
"AWS4-HMAC-SHA256\n{}\n{}\n{}", "AWS4-HMAC-SHA256\n{}\n{}\n{}",
@@ -355,8 +353,7 @@ impl StorageBackend for S3Storage {
canonical_uri, canonical_headers, signed_headers, payload_hash canonical_uri, canonical_headers, signed_headers, payload_hash
); );
let canonical_request_hash = let canonical_request_hash = hex::encode(Sha256::digest(canonical_request.as_bytes()));
hex::encode(sha2::Sha256::digest(canonical_request.as_bytes()));
let credential_scope = format!("{}/{}/s3/aws4_request", date, self.region); let credential_scope = format!("{}/{}/s3/aws4_request", date, self.region);
let string_to_sign = format!( let string_to_sign = format!(
"AWS4-HMAC-SHA256\n{}\n{}\n{}", "AWS4-HMAC-SHA256\n{}\n{}\n{}",
@@ -382,17 +379,6 @@ impl StorageBackend for S3Storage {
} }
} }
async fn total_size(&self) -> u64 {
let keys = self.list("").await;
let mut total = 0u64;
for key in &keys {
if let Some(meta) = self.stat(key).await {
total += meta.size;
}
}
total
}
fn backend_name(&self) -> &'static str { fn backend_name(&self) -> &'static str {
"s3" "s3"
} }
@@ -438,48 +424,4 @@ mod tests {
let result = hmac_sha256(b"key", b"data"); let result = hmac_sha256(b"key", b"data");
assert!(!result.is_empty()); assert!(!result.is_empty());
} }
#[test]
fn test_uri_encode_safe_chars() {
assert_eq!(uri_encode("hello"), "hello");
assert_eq!(uri_encode("foo/bar"), "foo/bar");
assert_eq!(uri_encode("test-file_v1.0"), "test-file_v1.0");
assert_eq!(uri_encode("a~b"), "a~b");
}
#[test]
fn test_uri_encode_special_chars() {
assert_eq!(uri_encode("hello world"), "hello%20world");
assert_eq!(uri_encode("file name.txt"), "file%20name.txt");
}
#[test]
fn test_uri_encode_query_chars() {
assert_eq!(uri_encode("key=value"), "key%3Dvalue");
assert_eq!(uri_encode("a&b"), "a%26b");
assert_eq!(uri_encode("a+b"), "a%2Bb");
}
#[test]
fn test_uri_encode_empty() {
assert_eq!(uri_encode(""), "");
}
#[test]
fn test_uri_encode_all_safe_ranges() {
// A-Z
assert_eq!(uri_encode("ABCXYZ"), "ABCXYZ");
// a-z
assert_eq!(uri_encode("abcxyz"), "abcxyz");
// 0-9
assert_eq!(uri_encode("0123456789"), "0123456789");
// Special safe: - _ . ~ /
assert_eq!(uri_encode("-_.~/"), "-_.~/");
}
#[test]
fn test_uri_encode_percent() {
assert_eq!(uri_encode("%"), "%25");
assert_eq!(uri_encode("100%done"), "100%25done");
}
} }

View File

@@ -1,260 +0,0 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT
//! Shared test infrastructure for integration tests.
//!
//! Provides `TestContext` that builds a full axum Router backed by a
//! tempdir-based local storage with all upstream proxies disabled.
#![allow(clippy::unwrap_used)] // tests may use .unwrap() freely
use axum::{body::Body, extract::DefaultBodyLimit, http::Request, middleware, Router};
use http_body_util::BodyExt;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Instant;
use tempfile::TempDir;
use crate::activity_log::ActivityLog;
use crate::audit::AuditLog;
use crate::auth::HtpasswdAuth;
use crate::config::*;
use crate::dashboard_metrics::DashboardMetrics;
use crate::registry;
use crate::repo_index::RepoIndex;
use crate::storage::Storage;
use crate::tokens::TokenStore;
use crate::AppState;
use parking_lot::RwLock;
/// Everything a test needs: tempdir (must stay alive), shared state, and the router.
pub struct TestContext {
pub state: Arc<AppState>,
pub app: Router,
pub _tempdir: TempDir,
}
/// Build a test context with auth **disabled** and all proxies off.
pub fn create_test_context() -> TestContext {
build_context(false, &[], false, |_| {})
}
/// Build a test context with auth **enabled** (bcrypt cost=4 for speed).
pub fn create_test_context_with_auth(users: &[(&str, &str)]) -> TestContext {
build_context(true, users, false, |_| {})
}
/// Build a test context with auth + anonymous_read.
pub fn create_test_context_with_anonymous_read(users: &[(&str, &str)]) -> TestContext {
build_context(true, users, true, |_| {})
}
/// Build a test context with raw storage **disabled**.
pub fn create_test_context_with_raw_disabled() -> TestContext {
build_context(false, &[], false, |cfg| cfg.raw.enabled = false)
}
fn build_context(
auth_enabled: bool,
users: &[(&str, &str)],
anonymous_read: bool,
customize: impl FnOnce(&mut Config),
) -> TestContext {
let tempdir = TempDir::new().expect("failed to create tempdir");
let storage_path = tempdir.path().to_str().unwrap().to_string();
let mut config = Config {
server: ServerConfig {
host: "127.0.0.1".into(),
port: 0,
public_url: None,
body_limit_mb: 2048,
},
storage: StorageConfig {
mode: StorageMode::Local,
path: storage_path.clone(),
s3_url: String::new(),
bucket: String::new(),
s3_access_key: None,
s3_secret_key: None,
s3_region: String::new(),
},
maven: MavenConfig {
proxies: vec![],
proxy_timeout: 5,
},
npm: NpmConfig {
proxy: None,
proxy_auth: None,
proxy_timeout: 5,
metadata_ttl: 0,
},
pypi: PypiConfig {
proxy: None,
proxy_auth: None,
proxy_timeout: 5,
},
go: GoConfig {
proxy: None,
proxy_auth: None,
proxy_timeout: 5,
proxy_timeout_zip: 30,
max_zip_size: 10_485_760,
},
cargo: CargoConfig {
proxy: None,
proxy_auth: None,
proxy_timeout: 5,
},
docker: DockerConfig {
proxy_timeout: 5,
upstreams: vec![],
},
raw: RawConfig {
enabled: true,
max_file_size: 1_048_576, // 1 MB
},
auth: AuthConfig {
enabled: auth_enabled,
anonymous_read,
htpasswd_file: String::new(),
token_storage: tempdir.path().join("tokens").to_str().unwrap().to_string(),
},
rate_limit: RateLimitConfig {
enabled: false,
..RateLimitConfig::default()
},
secrets: SecretsConfig::default(),
};
// Apply any custom config tweaks
customize(&mut config);
let storage = Storage::new_local(&storage_path);
let auth = if auth_enabled && !users.is_empty() {
let htpasswd_path = tempdir.path().join("users.htpasswd");
let mut content = String::new();
for (username, password) in users {
let hash = bcrypt::hash(password, 4).expect("bcrypt hash");
content.push_str(&format!("{}:{}\n", username, hash));
}
std::fs::write(&htpasswd_path, &content).expect("write htpasswd");
config.auth.htpasswd_file = htpasswd_path.to_str().unwrap().to_string();
HtpasswdAuth::from_file(&htpasswd_path)
} else {
None
};
let tokens = if auth_enabled {
Some(TokenStore::new(tempdir.path().join("tokens").as_path()))
} else {
None
};
let docker_auth = registry::DockerAuth::new(config.docker.proxy_timeout);
let state = Arc::new(AppState {
storage,
config,
start_time: Instant::now(),
auth,
tokens,
metrics: DashboardMetrics::new(),
activity: ActivityLog::new(50),
audit: AuditLog::new(&storage_path),
docker_auth,
repo_index: RepoIndex::new(),
http_client: reqwest::Client::new(),
upload_sessions: Arc::new(RwLock::new(HashMap::new())),
});
// Build router identical to run_server() but without TcpListener / rate-limiting
let registry_routes = Router::new()
.merge(registry::docker_routes())
.merge(registry::maven_routes())
.merge(registry::npm_routes())
.merge(registry::cargo_routes())
.merge(registry::pypi_routes())
.merge(registry::raw_routes())
.merge(registry::go_routes());
let public_routes = Router::new().merge(crate::health::routes());
let app_routes = Router::new()
.merge(crate::auth::token_routes())
.merge(registry_routes);
let app = Router::new()
.merge(public_routes)
.merge(app_routes)
.layer(DefaultBodyLimit::max(
state.config.server.body_limit_mb * 1024 * 1024,
))
.layer(middleware::from_fn(
crate::request_id::request_id_middleware,
))
.layer(middleware::from_fn_with_state(
state.clone(),
crate::auth::auth_middleware,
))
.with_state(state.clone());
TestContext {
state,
app,
_tempdir: tempdir,
}
}
// ---------------------------------------------------------------------------
// Convenience helpers
// ---------------------------------------------------------------------------
/// Send a request through the router and return the response.
pub async fn send(
app: &Router,
method: axum::http::Method,
uri: &str,
body: impl Into<Body>,
) -> axum::http::Response<Body> {
use tower::ServiceExt;
let request = Request::builder()
.method(method)
.uri(uri)
.body(body.into())
.unwrap();
app.clone().oneshot(request).await.unwrap()
}
/// Send a request with custom headers.
pub async fn send_with_headers(
app: &Router,
method: axum::http::Method,
uri: &str,
headers: Vec<(&str, &str)>,
body: impl Into<Body>,
) -> axum::http::Response<Body> {
use tower::ServiceExt;
let mut builder = Request::builder().method(method).uri(uri);
for (k, v) in headers {
builder = builder.header(k, v);
}
let request = builder.body(body.into()).unwrap();
app.clone().oneshot(request).await.unwrap()
}
/// Read the full response body into bytes.
pub async fn body_bytes(response: axum::http::Response<Body>) -> axum::body::Bytes {
response
.into_body()
.collect()
.await
.expect("failed to read body")
.to_bytes()
}

View File

@@ -1,36 +1,14 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay // Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT // SPDX-License-Identifier: MIT
use argon2::{
password_hash::{rand_core::OsRng, PasswordHash, PasswordHasher, PasswordVerifier, SaltString},
Argon2,
};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use std::fs; use std::fs;
use std::os::unix::fs::PermissionsExt;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::time::{SystemTime, UNIX_EPOCH}; use std::time::{SystemTime, UNIX_EPOCH};
use thiserror::Error; use thiserror::Error;
use uuid::Uuid; use uuid::Uuid;
use parking_lot::RwLock;
use std::collections::HashMap;
use std::sync::Arc;
use std::time::{Duration, Instant};
/// TTL for cached token verifications (avoids Argon2 per request)
const CACHE_TTL: Duration = Duration::from_secs(300);
/// Cached verification result
#[derive(Clone)]
struct CachedToken {
user: String,
role: Role,
expires_at: u64,
cached_at: Instant,
}
const TOKEN_PREFIX: &str = "nra_"; const TOKEN_PREFIX: &str = "nra_";
/// Access role for API tokens /// Access role for API tokens
@@ -83,25 +61,15 @@ fn default_role() -> Role {
#[derive(Clone)] #[derive(Clone)]
pub struct TokenStore { pub struct TokenStore {
storage_path: PathBuf, storage_path: PathBuf,
/// In-memory cache: SHA256(token) -> verified result (avoids Argon2 per request)
cache: Arc<RwLock<HashMap<String, CachedToken>>>,
/// Pending last_used updates: file_id_prefix -> timestamp (flushed periodically)
pending_last_used: Arc<RwLock<HashMap<String, u64>>>,
} }
impl TokenStore { impl TokenStore {
/// Create a new token store /// Create a new token store
pub fn new(storage_path: &Path) -> Self { pub fn new(storage_path: &Path) -> Self {
// Ensure directory exists with restricted permissions // Ensure directory exists
let _ = fs::create_dir_all(storage_path); let _ = fs::create_dir_all(storage_path);
#[cfg(unix)]
{
let _ = fs::set_permissions(storage_path, fs::Permissions::from_mode(0o700));
}
Self { Self {
storage_path: storage_path.to_path_buf(), storage_path: storage_path.to_path_buf(),
cache: Arc::new(RwLock::new(HashMap::new())),
pending_last_used: Arc::new(RwLock::new(HashMap::new())),
} }
} }
@@ -119,9 +87,7 @@ impl TokenStore {
TOKEN_PREFIX, TOKEN_PREFIX,
Uuid::new_v4().to_string().replace("-", "") Uuid::new_v4().to_string().replace("-", "")
); );
let token_hash = hash_token_argon2(&raw_token)?; let token_hash = hash_token(&raw_token);
// Use SHA256 of token as filename (deterministic, for lookup)
let file_id = sha256_hex(&raw_token);
let now = SystemTime::now() let now = SystemTime::now()
.duration_since(UNIX_EPOCH) .duration_since(UNIX_EPOCH)
@@ -131,7 +97,7 @@ impl TokenStore {
let expires_at = now + (ttl_days * 24 * 60 * 60); let expires_at = now + (ttl_days * 24 * 60 * 60);
let info = TokenInfo { let info = TokenInfo {
token_hash, token_hash: token_hash.clone(),
user: user.to_string(), user: user.to_string(),
created_at: now, created_at: now,
expires_at, expires_at,
@@ -140,84 +106,39 @@ impl TokenStore {
role, role,
}; };
// Save to file with restricted permissions // Save to file
let file_path = self.storage_path.join(format!("{}.json", &file_id[..16])); let file_path = self
.storage_path
.join(format!("{}.json", &token_hash[..16]));
let json = let json =
serde_json::to_string_pretty(&info).map_err(|e| TokenError::Storage(e.to_string()))?; serde_json::to_string_pretty(&info).map_err(|e| TokenError::Storage(e.to_string()))?;
fs::write(&file_path, &json).map_err(|e| TokenError::Storage(e.to_string()))?; fs::write(&file_path, json).map_err(|e| TokenError::Storage(e.to_string()))?;
set_file_permissions_600(&file_path);
Ok(raw_token) Ok(raw_token)
} }
/// Verify a token and return user info if valid. /// Verify a token and return user info if valid
///
/// Uses an in-memory cache to avoid Argon2 verification on every request.
/// The `last_used` timestamp is updated in batch via `flush_last_used()`.
pub fn verify_token(&self, token: &str) -> Result<(String, Role), TokenError> { pub fn verify_token(&self, token: &str) -> Result<(String, Role), TokenError> {
if !token.starts_with(TOKEN_PREFIX) { if !token.starts_with(TOKEN_PREFIX) {
return Err(TokenError::InvalidFormat); return Err(TokenError::InvalidFormat);
} }
let cache_key = sha256_hex(token); let token_hash = hash_token(token);
let file_path = self
.storage_path
.join(format!("{}.json", &token_hash[..16]));
// Fast path: check in-memory cache if !file_path.exists() {
{ return Err(TokenError::NotFound);
let cache = self.cache.read();
if let Some(cached) = cache.get(&cache_key) {
if cached.cached_at.elapsed() < CACHE_TTL {
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs();
if now > cached.expires_at {
return Err(TokenError::Expired);
}
// Schedule deferred last_used update
self.pending_last_used
.write()
.insert(cache_key[..16].to_string(), now);
return Ok((cached.user.clone(), cached.role.clone()));
}
}
} }
// Slow path: read from disk and verify Argon2 let content =
let file_path = self.storage_path.join(format!("{}.json", &cache_key[..16])); fs::read_to_string(&file_path).map_err(|e| TokenError::Storage(e.to_string()))?;
let content = match fs::read_to_string(&file_path) {
Ok(c) => c,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
return Err(TokenError::NotFound);
}
Err(e) => return Err(TokenError::Storage(e.to_string())),
};
let mut info: TokenInfo = let mut info: TokenInfo =
serde_json::from_str(&content).map_err(|e| TokenError::Storage(e.to_string()))?; serde_json::from_str(&content).map_err(|e| TokenError::Storage(e.to_string()))?;
// Verify hash: try Argon2id first, fall back to legacy SHA256 // Verify hash matches
let hash_valid = if info.token_hash.starts_with("$argon2") { if info.token_hash != token_hash {
verify_token_argon2(token, &info.token_hash)
} else {
// Legacy SHA256 hash (no salt) — verify and migrate
let legacy_hash = sha256_hex(token);
if info.token_hash == legacy_hash {
// Migrate to Argon2id
if let Ok(new_hash) = hash_token_argon2(token) {
info.token_hash = new_hash;
if let Ok(json) = serde_json::to_string_pretty(&info) {
let _ = fs::write(&file_path, &json);
set_file_permissions_600(&file_path);
}
}
true
} else {
false
}
};
if !hash_valid {
return Err(TokenError::NotFound); return Err(TokenError::NotFound);
} }
@@ -231,21 +152,11 @@ impl TokenStore {
return Err(TokenError::Expired); return Err(TokenError::Expired);
} }
// Populate cache // Update last_used
self.cache.write().insert( info.last_used = Some(now);
cache_key[..16].to_string(), if let Ok(json) = serde_json::to_string_pretty(&info) {
CachedToken { let _ = fs::write(&file_path, json);
user: info.user.clone(), }
role: info.role.clone(),
expires_at: info.expires_at,
cached_at: Instant::now(),
},
);
// Schedule deferred last_used update
self.pending_last_used
.write()
.insert(cache_key[..16].to_string(), now);
Ok((info.user, info.role)) Ok((info.user, info.role))
} }
@@ -270,56 +181,17 @@ impl TokenStore {
tokens tokens
} }
/// Flush pending last_used timestamps to disk (async to avoid blocking runtime).
/// Called periodically by background task (every 30s).
pub async fn flush_last_used(&self) {
let pending: HashMap<String, u64> = {
let mut map = self.pending_last_used.write();
std::mem::take(&mut *map)
};
if pending.is_empty() {
return;
}
for (file_prefix, timestamp) in &pending {
let file_path = self.storage_path.join(format!("{}.json", file_prefix));
let content = match tokio::fs::read_to_string(&file_path).await {
Ok(c) => c,
Err(_) => continue,
};
let mut info: TokenInfo = match serde_json::from_str(&content) {
Ok(i) => i,
Err(_) => continue,
};
info.last_used = Some(*timestamp);
if let Ok(json) = serde_json::to_string_pretty(&info) {
let _ = tokio::fs::write(&file_path, &json).await;
set_file_permissions_600(&file_path);
}
}
tracing::debug!(count = pending.len(), "Flushed pending last_used updates");
}
/// Remove a token from the in-memory cache (called on revoke)
fn invalidate_cache(&self, hash_prefix: &str) {
self.cache.write().remove(hash_prefix);
}
/// Revoke a token by its hash prefix /// Revoke a token by its hash prefix
pub fn revoke_token(&self, hash_prefix: &str) -> Result<(), TokenError> { pub fn revoke_token(&self, hash_prefix: &str) -> Result<(), TokenError> {
let file_path = self.storage_path.join(format!("{}.json", hash_prefix)); let file_path = self.storage_path.join(format!("{}.json", hash_prefix));
// TOCTOU fix: try remove directly if !file_path.exists() {
match fs::remove_file(&file_path) { return Err(TokenError::NotFound);
Ok(()) => {
self.invalidate_cache(hash_prefix);
Ok(())
}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Err(TokenError::NotFound),
Err(e) => Err(TokenError::Storage(e.to_string())),
} }
fs::remove_file(&file_path).map_err(|e| TokenError::Storage(e.to_string()))?;
Ok(())
} }
/// Revoke all tokens for a user /// Revoke all tokens for a user
@@ -342,39 +214,11 @@ impl TokenStore {
} }
} }
/// Hash a token using Argon2id with random salt /// Hash a token using SHA256
fn hash_token_argon2(token: &str) -> Result<String, TokenError> { fn hash_token(token: &str) -> String {
let salt = SaltString::generate(&mut OsRng);
let argon2 = Argon2::default();
argon2
.hash_password(token.as_bytes(), &salt)
.map(|h| h.to_string())
.map_err(|e| TokenError::Storage(format!("hash error: {e}")))
}
/// Verify a token against an Argon2id hash
fn verify_token_argon2(token: &str, hash: &str) -> bool {
match PasswordHash::new(hash) {
Ok(parsed) => Argon2::default()
.verify_password(token.as_bytes(), &parsed)
.is_ok(),
Err(_) => false,
}
}
/// SHA256 hex digest (used for file naming and legacy hash verification)
fn sha256_hex(input: &str) -> String {
let mut hasher = Sha256::new(); let mut hasher = Sha256::new();
hasher.update(input.as_bytes()); hasher.update(token.as_bytes());
hex::encode(hasher.finalize()) format!("{:x}", hasher.finalize())
}
/// Set file permissions to 600 (owner read/write only)
fn set_file_permissions_600(path: &Path) {
#[cfg(unix)]
{
let _ = fs::set_permissions(path, fs::Permissions::from_mode(0o600));
}
} }
#[derive(Debug, Error)] #[derive(Debug, Error)]
@@ -393,7 +237,6 @@ pub enum TokenError {
} }
#[cfg(test)] #[cfg(test)]
#[allow(clippy::unwrap_used)]
mod tests { mod tests {
use super::*; use super::*;
use tempfile::TempDir; use tempfile::TempDir;
@@ -411,19 +254,6 @@ mod tests {
assert_eq!(token.len(), 4 + 32); // prefix + uuid without dashes assert_eq!(token.len(), 4 + 32); // prefix + uuid without dashes
} }
#[test]
fn test_token_hash_is_argon2() {
let temp_dir = TempDir::new().unwrap();
let store = TokenStore::new(temp_dir.path());
let _token = store
.create_token("testuser", 30, None, Role::Write)
.unwrap();
let tokens = store.list_tokens("testuser");
assert!(tokens[0].token_hash.starts_with("$argon2"));
}
#[test] #[test]
fn test_verify_valid_token() { fn test_verify_valid_token() {
let temp_dir = TempDir::new().unwrap(); let temp_dir = TempDir::new().unwrap();
@@ -461,80 +291,24 @@ mod tests {
let temp_dir = TempDir::new().unwrap(); let temp_dir = TempDir::new().unwrap();
let store = TokenStore::new(temp_dir.path()); let store = TokenStore::new(temp_dir.path());
// Create token and manually set it as expired
let token = store let token = store
.create_token("testuser", 1, None, Role::Write) .create_token("testuser", 1, None, Role::Write)
.unwrap(); .unwrap();
let file_id = sha256_hex(&token); let token_hash = hash_token(&token);
let file_path = temp_dir.path().join(format!("{}.json", &file_id[..16])); let file_path = temp_dir.path().join(format!("{}.json", &token_hash[..16]));
// Read and modify the token to be expired
let content = std::fs::read_to_string(&file_path).unwrap(); let content = std::fs::read_to_string(&file_path).unwrap();
let mut info: TokenInfo = serde_json::from_str(&content).unwrap(); let mut info: TokenInfo = serde_json::from_str(&content).unwrap();
info.expires_at = 0; info.expires_at = 0; // Set to epoch (definitely expired)
std::fs::write(&file_path, serde_json::to_string(&info).unwrap()).unwrap(); std::fs::write(&file_path, serde_json::to_string(&info).unwrap()).unwrap();
// Token should now be expired
let result = store.verify_token(&token); let result = store.verify_token(&token);
assert!(matches!(result, Err(TokenError::Expired))); assert!(matches!(result, Err(TokenError::Expired)));
} }
#[test]
fn test_legacy_sha256_migration() {
let temp_dir = TempDir::new().unwrap();
let store = TokenStore::new(temp_dir.path());
// Simulate a legacy token with SHA256 hash
let raw_token = "nra_00112233445566778899aabbccddeeff";
let legacy_hash = sha256_hex(raw_token);
let file_id = sha256_hex(raw_token);
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_default()
.as_secs();
let info = TokenInfo {
token_hash: legacy_hash.clone(),
user: "legacyuser".to_string(),
created_at: now,
expires_at: now + 86400,
last_used: None,
description: None,
role: Role::Read,
};
let file_path = temp_dir.path().join(format!("{}.json", &file_id[..16]));
fs::write(&file_path, serde_json::to_string_pretty(&info).unwrap()).unwrap();
// Verify should work with legacy hash
let (user, role) = store.verify_token(raw_token).unwrap();
assert_eq!(user, "legacyuser");
assert_eq!(role, Role::Read);
// After verification, hash should be migrated to Argon2id
let content = fs::read_to_string(&file_path).unwrap();
let updated: TokenInfo = serde_json::from_str(&content).unwrap();
assert!(updated.token_hash.starts_with("$argon2"));
}
#[test]
fn test_file_permissions() {
let temp_dir = TempDir::new().unwrap();
let store = TokenStore::new(temp_dir.path());
let token = store
.create_token("testuser", 30, None, Role::Write)
.unwrap();
let file_id = sha256_hex(&token);
let file_path = temp_dir.path().join(format!("{}.json", &file_id[..16]));
#[cfg(unix)]
{
let metadata = fs::metadata(&file_path).unwrap();
let mode = metadata.permissions().mode() & 0o777;
assert_eq!(mode, 0o600);
}
}
#[test] #[test]
fn test_list_tokens() { fn test_list_tokens() {
let temp_dir = TempDir::new().unwrap(); let temp_dir = TempDir::new().unwrap();
@@ -562,13 +336,16 @@ mod tests {
let token = store let token = store
.create_token("testuser", 30, None, Role::Write) .create_token("testuser", 30, None, Role::Write)
.unwrap(); .unwrap();
let file_id = sha256_hex(&token); let token_hash = hash_token(&token);
let hash_prefix = &file_id[..16]; let hash_prefix = &token_hash[..16];
// Verify token works
assert!(store.verify_token(&token).is_ok()); assert!(store.verify_token(&token).is_ok());
// Revoke
store.revoke_token(hash_prefix).unwrap(); store.revoke_token(hash_prefix).unwrap();
// Verify token no longer works
let result = store.verify_token(&token); let result = store.verify_token(&token);
assert!(matches!(result, Err(TokenError::NotFound))); assert!(matches!(result, Err(TokenError::NotFound)));
} }
@@ -598,8 +375,8 @@ mod tests {
assert_eq!(store.list_tokens("user2").len(), 1); assert_eq!(store.list_tokens("user2").len(), 1);
} }
#[tokio::test] #[test]
async fn test_token_updates_last_used() { fn test_token_updates_last_used() {
let temp_dir = TempDir::new().unwrap(); let temp_dir = TempDir::new().unwrap();
let store = TokenStore::new(temp_dir.path()); let store = TokenStore::new(temp_dir.path());
@@ -607,57 +384,14 @@ mod tests {
.create_token("testuser", 30, None, Role::Write) .create_token("testuser", 30, None, Role::Write)
.unwrap(); .unwrap();
// First verification
store.verify_token(&token).unwrap(); store.verify_token(&token).unwrap();
// last_used is deferred — flush to persist // Check last_used is set
store.flush_last_used().await;
let tokens = store.list_tokens("testuser"); let tokens = store.list_tokens("testuser");
assert!(tokens[0].last_used.is_some()); assert!(tokens[0].last_used.is_some());
} }
#[test]
fn test_verify_cache_hit() {
let temp_dir = TempDir::new().unwrap();
let store = TokenStore::new(temp_dir.path());
let token = store
.create_token("testuser", 30, None, Role::Write)
.unwrap();
// First call: cold (disk + Argon2)
let (user1, role1) = store.verify_token(&token).unwrap();
// Second call: should hit cache (no Argon2)
let (user2, role2) = store.verify_token(&token).unwrap();
assert_eq!(user1, user2);
assert_eq!(role1, role2);
assert_eq!(user1, "testuser");
assert_eq!(role1, Role::Write);
}
#[test]
fn test_revoke_invalidates_cache() {
let temp_dir = TempDir::new().unwrap();
let store = TokenStore::new(temp_dir.path());
let token = store
.create_token("testuser", 30, None, Role::Write)
.unwrap();
let file_id = sha256_hex(&token);
let hash_prefix = &file_id[..16];
// Populate cache
assert!(store.verify_token(&token).is_ok());
// Revoke
store.revoke_token(hash_prefix).unwrap();
// Cache should be invalidated
let result = store.verify_token(&token);
assert!(matches!(result, Err(TokenError::NotFound)));
}
#[test] #[test]
fn test_token_with_description() { fn test_token_with_description() {
let temp_dir = TempDir::new().unwrap(); let temp_dir = TempDir::new().unwrap();

View File

@@ -23,8 +23,6 @@ pub struct RegistryStats {
pub npm: usize, pub npm: usize,
pub cargo: usize, pub cargo: usize,
pub pypi: usize, pub pypi: usize,
pub go: usize,
pub raw: usize,
} }
#[derive(Serialize)] #[derive(Serialize)]
@@ -116,18 +114,14 @@ pub async fn api_stats(State(state): State<Arc<AppState>>) -> Json<RegistryStats
let _ = state.repo_index.get("npm", &state.storage).await; let _ = state.repo_index.get("npm", &state.storage).await;
let _ = state.repo_index.get("cargo", &state.storage).await; let _ = state.repo_index.get("cargo", &state.storage).await;
let _ = state.repo_index.get("pypi", &state.storage).await; let _ = state.repo_index.get("pypi", &state.storage).await;
let _ = state.repo_index.get("go", &state.storage).await;
let _ = state.repo_index.get("raw", &state.storage).await;
let (docker, maven, npm, cargo, pypi, go, raw) = state.repo_index.counts(); let (docker, maven, npm, cargo, pypi) = state.repo_index.counts();
Json(RegistryStats { Json(RegistryStats {
docker, docker,
maven, maven,
npm, npm,
cargo, cargo,
pypi, pypi,
go,
raw,
}) })
} }
@@ -138,8 +132,6 @@ pub async fn api_dashboard(State(state): State<Arc<AppState>>) -> Json<Dashboard
let npm_repos = state.repo_index.get("npm", &state.storage).await; let npm_repos = state.repo_index.get("npm", &state.storage).await;
let cargo_repos = state.repo_index.get("cargo", &state.storage).await; let cargo_repos = state.repo_index.get("cargo", &state.storage).await;
let pypi_repos = state.repo_index.get("pypi", &state.storage).await; let pypi_repos = state.repo_index.get("pypi", &state.storage).await;
let go_repos = state.repo_index.get("go", &state.storage).await;
let raw_repos = state.repo_index.get("raw", &state.storage).await;
// Calculate sizes from cached index // Calculate sizes from cached index
let docker_size: u64 = docker_repos.iter().map(|r| r.size).sum(); let docker_size: u64 = docker_repos.iter().map(|r| r.size).sum();
@@ -147,10 +139,7 @@ pub async fn api_dashboard(State(state): State<Arc<AppState>>) -> Json<Dashboard
let npm_size: u64 = npm_repos.iter().map(|r| r.size).sum(); let npm_size: u64 = npm_repos.iter().map(|r| r.size).sum();
let cargo_size: u64 = cargo_repos.iter().map(|r| r.size).sum(); let cargo_size: u64 = cargo_repos.iter().map(|r| r.size).sum();
let pypi_size: u64 = pypi_repos.iter().map(|r| r.size).sum(); let pypi_size: u64 = pypi_repos.iter().map(|r| r.size).sum();
let go_size: u64 = go_repos.iter().map(|r| r.size).sum(); let total_storage = docker_size + maven_size + npm_size + cargo_size + pypi_size;
let raw_size: u64 = raw_repos.iter().map(|r| r.size).sum();
let total_storage =
docker_size + maven_size + npm_size + cargo_size + pypi_size + go_size + raw_size;
// Count total versions/tags, not just repositories // Count total versions/tags, not just repositories
let docker_versions: usize = docker_repos.iter().map(|r| r.versions).sum(); let docker_versions: usize = docker_repos.iter().map(|r| r.versions).sum();
@@ -158,15 +147,8 @@ pub async fn api_dashboard(State(state): State<Arc<AppState>>) -> Json<Dashboard
let npm_versions: usize = npm_repos.iter().map(|r| r.versions).sum(); let npm_versions: usize = npm_repos.iter().map(|r| r.versions).sum();
let cargo_versions: usize = cargo_repos.iter().map(|r| r.versions).sum(); let cargo_versions: usize = cargo_repos.iter().map(|r| r.versions).sum();
let pypi_versions: usize = pypi_repos.iter().map(|r| r.versions).sum(); let pypi_versions: usize = pypi_repos.iter().map(|r| r.versions).sum();
let go_versions: usize = go_repos.iter().map(|r| r.versions).sum(); let total_artifacts =
let raw_versions: usize = raw_repos.iter().map(|r| r.versions).sum(); docker_versions + maven_versions + npm_versions + cargo_versions + pypi_versions;
let total_artifacts = docker_versions
+ maven_versions
+ npm_versions
+ cargo_versions
+ pypi_versions
+ go_versions
+ raw_versions;
let global_stats = GlobalStats { let global_stats = GlobalStats {
downloads: state.metrics.downloads.load(Ordering::Relaxed), downloads: state.metrics.downloads.load(Ordering::Relaxed),
@@ -212,20 +194,6 @@ pub async fn api_dashboard(State(state): State<Arc<AppState>>) -> Json<Dashboard
uploads: 0, uploads: 0,
size_bytes: pypi_size, size_bytes: pypi_size,
}, },
RegistryCardStats {
name: "go".to_string(),
artifact_count: go_versions,
downloads: state.metrics.get_registry_downloads("go"),
uploads: 0,
size_bytes: go_size,
},
RegistryCardStats {
name: "raw".to_string(),
artifact_count: raw_versions,
downloads: state.metrics.get_registry_downloads("raw"),
uploads: state.metrics.get_registry_uploads("raw"),
size_bytes: raw_size,
},
]; ];
let mount_points = vec![ let mount_points = vec![
@@ -259,16 +227,6 @@ pub async fn api_dashboard(State(state): State<Arc<AppState>>) -> Json<Dashboard
mount_path: "/simple/".to_string(), mount_path: "/simple/".to_string(),
proxy_upstream: state.config.pypi.proxy.clone(), proxy_upstream: state.config.pypi.proxy.clone(),
}, },
MountPoint {
registry: "Go".to_string(),
mount_path: "/go/".to_string(),
proxy_upstream: state.config.go.proxy.clone(),
},
MountPoint {
registry: "Raw".to_string(),
mount_path: "/raw/".to_string(),
proxy_upstream: None,
},
]; ];
let activity = state.activity.recent(20); let activity = state.activity.recent(20);
@@ -415,32 +373,12 @@ pub async fn get_registry_stats(storage: &Storage) -> RegistryStats {
.collect::<HashSet<_>>() .collect::<HashSet<_>>()
.len(); .len();
let go = all_keys
.iter()
.filter(|k| k.starts_with("go/") && k.ends_with(".zip"))
.filter_map(|k| {
let rest = k.strip_prefix("go/")?;
let pos = rest.rfind("/@v/")?;
Some(rest[..pos].to_string())
})
.collect::<HashSet<_>>()
.len();
let raw = all_keys
.iter()
.filter(|k| k.starts_with("raw/"))
.filter_map(|k| k.strip_prefix("raw/")?.split('/').next())
.collect::<HashSet<_>>()
.len();
RegistryStats { RegistryStats {
docker, docker,
maven, maven,
npm, npm,
cargo, cargo,
pypi, pypi,
go,
raw,
} }
} }
@@ -936,32 +874,6 @@ pub async fn get_pypi_detail(storage: &Storage, name: &str) -> PackageDetail {
PackageDetail { versions } PackageDetail { versions }
} }
pub async fn get_go_detail(storage: &Storage, module: &str) -> PackageDetail {
let prefix = format!("go/{}/@v/", module);
let keys = storage.list(&prefix).await;
let mut versions = Vec::new();
for key in keys.iter().filter(|k| k.ends_with(".zip")) {
if let Some(rest) = key.strip_prefix(&prefix) {
if let Some(version) = rest.strip_suffix(".zip") {
let (size, published) = if let Some(meta) = storage.stat(key).await {
(meta.size, format_timestamp(meta.modified))
} else {
(0, "N/A".to_string())
};
versions.push(VersionInfo {
version: version.to_string(),
size,
published,
});
}
}
}
versions.sort_by(|a, b| b.version.cmp(&a.version));
PackageDetail { versions }
}
fn extract_pypi_version(name: &str, filename: &str) -> Option<String> { fn extract_pypi_version(name: &str, filename: &str) -> Option<String> {
// Handle both .tar.gz and .whl files // Handle both .tar.gz and .whl files
let clean_name = name.replace('-', "_"); let clean_name = name.replace('-', "_");
@@ -985,26 +897,3 @@ fn extract_pypi_version(name: &str, filename: &str) -> Option<String> {
None None
} }
} }
pub async fn get_raw_detail(storage: &Storage, group: &str) -> PackageDetail {
let prefix = format!("raw/{}/", group);
let keys = storage.list(&prefix).await;
let mut versions = Vec::new();
for key in &keys {
if let Some(filename) = key.strip_prefix(&prefix) {
let (size, published) = if let Some(meta) = storage.stat(key).await {
(meta.size, format_timestamp(meta.modified))
} else {
(0, "N/A".to_string())
};
versions.push(VersionInfo {
version: filename.to_string(),
size,
published,
});
}
}
PackageDetail { versions }
}

View File

@@ -90,7 +90,7 @@ fn sidebar_dark(active_page: Option<&str>, t: &Translations) -> String {
let docker_icon = r#"<path fill="currentColor" d="M13.983 11.078h2.119a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.119a.185.185 0 00-.185.185v1.888c0 .102.083.185.185.185m-2.954-5.43h2.118a.186.186 0 00.186-.186V3.574a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.186m0 2.716h2.118a.187.187 0 00.186-.186V6.29a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.887c0 .102.082.185.185.186m-2.93 0h2.12a.186.186 0 00.184-.186V6.29a.185.185 0 00-.185-.185H8.1a.185.185 0 00-.185.185v1.887c0 .102.083.185.185.186m-2.964 0h2.119a.186.186 0 00.185-.186V6.29a.185.185 0 00-.185-.185H5.136a.186.186 0 00-.186.185v1.887c0 .102.084.185.186.186m5.893 2.715h2.118a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.185m-2.93 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.083.185.185.185m-2.964 0h2.119a.185.185 0 00.185-.185V9.006a.185.185 0 00-.185-.186h-2.12a.186.186 0 00-.185.186v1.887c0 .102.084.185.186.185m-2.92 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.082.185.185.185M23.763 9.89c-.065-.051-.672-.51-1.954-.51-.338.001-.676.03-1.01.087-.248-1.7-1.653-2.53-1.716-2.566l-.344-.199-.226.327c-.284.438-.49.922-.612 1.43-.23.97-.09 1.882.403 2.661-.595.332-1.55.413-1.744.42H.751a.751.751 0 00-.75.748 11.376 11.376 0 00.692 4.062c.545 1.428 1.355 2.48 2.41 3.124 1.18.723 3.1 1.137 5.275 1.137.983.003 1.963-.086 2.93-.266a12.248 12.248 0 003.823-1.389c.98-.567 1.86-1.288 2.61-2.136 1.252-1.418 1.998-2.997 2.553-4.4h.221c1.372 0 2.215-.549 2.68-1.009.309-.293.55-.65.707-1.046l.098-.288Z"/>"#; let docker_icon = r#"<path fill="currentColor" d="M13.983 11.078h2.119a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.119a.185.185 0 00-.185.185v1.888c0 .102.083.185.185.185m-2.954-5.43h2.118a.186.186 0 00.186-.186V3.574a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.186m0 2.716h2.118a.187.187 0 00.186-.186V6.29a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.887c0 .102.082.185.185.186m-2.93 0h2.12a.186.186 0 00.184-.186V6.29a.185.185 0 00-.185-.185H8.1a.185.185 0 00-.185.185v1.887c0 .102.083.185.185.186m-2.964 0h2.119a.186.186 0 00.185-.186V6.29a.185.185 0 00-.185-.185H5.136a.186.186 0 00-.186.185v1.887c0 .102.084.185.186.186m5.893 2.715h2.118a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.185m-2.93 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.083.185.185.185m-2.964 0h2.119a.185.185 0 00.185-.185V9.006a.185.185 0 00-.185-.186h-2.12a.186.186 0 00-.185.186v1.887c0 .102.084.185.186.185m-2.92 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.082.185.185.185M23.763 9.89c-.065-.051-.672-.51-1.954-.51-.338.001-.676.03-1.01.087-.248-1.7-1.653-2.53-1.716-2.566l-.344-.199-.226.327c-.284.438-.49.922-.612 1.43-.23.97-.09 1.882.403 2.661-.595.332-1.55.413-1.744.42H.751a.751.751 0 00-.75.748 11.376 11.376 0 00.692 4.062c.545 1.428 1.355 2.48 2.41 3.124 1.18.723 3.1 1.137 5.275 1.137.983.003 1.963-.086 2.93-.266a12.248 12.248 0 003.823-1.389c.98-.567 1.86-1.288 2.61-2.136 1.252-1.418 1.998-2.997 2.553-4.4h.221c1.372 0 2.215-.549 2.68-1.009.309-.293.55-.65.707-1.046l.098-.288Z"/>"#;
let maven_icon = r#"<path fill="currentColor" d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm-1 17.93c-3.95-.49-7-3.85-7-7.93 0-.62.08-1.21.21-1.79L9 15v1c0 1.1.9 2 2 2v1.93zm6.9-2.54c-.26-.81-1-1.39-1.9-1.39h-1v-3c0-.55-.45-1-1-1H8v-2h2c.55 0 1-.45 1-1V7h2c1.1 0 2-.9 2-2v-.41c2.93 1.19 5 4.06 5 7.41 0 2.08-.8 3.97-2.1 5.39z"/>"#; let maven_icon = r#"<path fill="currentColor" d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm-1 17.93c-3.95-.49-7-3.85-7-7.93 0-.62.08-1.21.21-1.79L9 15v1c0 1.1.9 2 2 2v1.93zm6.9-2.54c-.26-.81-1-1.39-1.9-1.39h-1v-3c0-.55-.45-1-1-1H8v-2h2c.55 0 1-.45 1-1V7h2c1.1 0 2-.9 2-2v-.41c2.93 1.19 5 4.06 5 7.41 0 2.08-.8 3.97-2.1 5.39z"/>"#;
let npm_icon = r#"<path fill="currentColor" d="M0 7.334v8h6.666v1.332H12v-1.332h12v-8H0zm6.666 6.664H5.334v-4H3.999v4H1.335V8.667h5.331v5.331zm4 0v1.336H8.001V8.667h5.334v5.332h-2.669v-.001zm12.001 0h-1.33v-4h-1.336v4h-1.335v-4h-1.33v4h-2.671V8.667h8.002v5.331zM10.665 10H12v2.667h-1.335V10z"/>"#; let npm_icon = r#"<path fill="currentColor" d="M0 7.334v8h6.666v1.332H12v-1.332h12v-8H0zm6.666 6.664H5.334v-4H3.999v4H1.335V8.667h5.331v5.331zm4 0v1.336H8.001V8.667h5.334v5.332h-2.669v-.001zm12.001 0h-1.33v-4h-1.336v4h-1.335v-4h-1.33v4h-2.671V8.667h8.002v5.331zM10.665 10H12v2.667h-1.335V10z"/>"#;
let cargo_icon = r#"<path fill="currentColor" d="M6 2h12a1 1 0 011 1v8a1 1 0 01-1 1H6a1 1 0 01-1-1V3a1 1 0 011-1zm0 2v2h12V4H6zm0 3v2h12V7H6zM2 14h8a1 1 0 011 1v6a1 1 0 01-1 1H2a1 1 0 01-1-1v-6a1 1 0 011-1zm0 2v1.5h8V16H2zM14 14h8a1 1 0 011 1v6a1 1 0 01-1 1h-8a1 1 0 01-1-1v-6a1 1 0 011-1zm0 2v1.5h8V16h-8z"/>"#; let cargo_icon = r#"<path fill="currentColor" d="M20 8h-3V4H3c-1.1 0-2 .9-2 2v11h2c0 1.66 1.34 3 3 3s3-1.34 3-3h6c0 1.66 1.34 3 3 3s3-1.34 3-3h2v-5l-3-4zM6 18.5c-.83 0-1.5-.67-1.5-1.5s.67-1.5 1.5-1.5 1.5.67 1.5 1.5-.67 1.5-1.5 1.5zm13.5-9l1.96 2.5H17V9.5h2.5zm-1.5 9c-.83 0-1.5-.67-1.5-1.5s.67-1.5 1.5-1.5 1.5.67 1.5 1.5-.67 1.5-1.5 1.5z"/>"#;
let pypi_icon = r#"<path fill="currentColor" d="M14.25.18l.9.2.73.26.59.3.45.32.34.34.25.34.16.33.1.3.04.26.02.2-.01.13V8.5l-.05.63-.13.55-.21.46-.26.38-.3.31-.33.25-.35.19-.35.14-.33.1-.3.07-.26.04-.21.02H8.83l-.69.05-.59.14-.5.22-.41.27-.33.32-.27.35-.2.36-.15.37-.1.35-.07.32-.04.27-.02.21v3.06H3.23l-.21-.03-.28-.07-.32-.12-.35-.18-.36-.26-.36-.36-.35-.46-.32-.59-.28-.73-.21-.88-.14-1.05L0 11.97l.06-1.22.16-1.04.24-.87.32-.71.36-.57.4-.44.42-.33.42-.24.4-.16.36-.1.32-.05.24-.01h.16l.06.01h8.16v-.83H6.24l-.01-2.75-.02-.37.05-.34.11-.31.17-.28.25-.26.31-.23.38-.2.44-.18.51-.15.58-.12.64-.1.71-.06.77-.04.84-.02 1.27.05 1.07.13zm-6.3 1.98l-.23.33-.08.41.08.41.23.34.33.22.41.09.41-.09.33-.22.23-.34.08-.41-.08-.41-.23-.33-.33-.22-.41-.09-.41.09-.33.22zM21.1 6.11l.28.06.32.12.35.18.36.27.36.35.35.47.32.59.28.73.21.88.14 1.04.05 1.23-.06 1.23-.16 1.04-.24.86-.32.71-.36.57-.4.45-.42.33-.42.24-.4.16-.36.09-.32.05-.24.02-.16-.01h-8.22v.82h5.84l.01 2.76.02.36-.05.34-.11.31-.17.29-.25.25-.31.24-.38.2-.44.17-.51.15-.58.13-.64.09-.71.07-.77.04-.84.01-1.27-.04-1.07-.14-.9-.2-.73-.25-.59-.3-.45-.33-.34-.34-.25-.34-.16-.33-.1-.3-.04-.25-.02-.2.01-.13v-5.34l.05-.64.13-.54.21-.46.26-.38.3-.32.33-.24.35-.2.35-.14.33-.1.3-.06.26-.04.21-.02.13-.01h5.84l.69-.05.59-.14.5-.21.41-.28.33-.32.27-.35.2-.36.15-.36.1-.35.07-.32.04-.28.02-.21V6.07h2.09l.14.01.21.03zm-6.47 14.25l-.23.33-.08.41.08.41.23.33.33.23.41.08.41-.08.33-.23.23-.33.08-.41-.08-.41-.23-.33-.33-.23-.41-.08-.41.08-.33.23z"/>"#; let pypi_icon = r#"<path fill="currentColor" d="M14.25.18l.9.2.73.26.59.3.45.32.34.34.25.34.16.33.1.3.04.26.02.2-.01.13V8.5l-.05.63-.13.55-.21.46-.26.38-.3.31-.33.25-.35.19-.35.14-.33.1-.3.07-.26.04-.21.02H8.83l-.69.05-.59.14-.5.22-.41.27-.33.32-.27.35-.2.36-.15.37-.1.35-.07.32-.04.27-.02.21v3.06H3.23l-.21-.03-.28-.07-.32-.12-.35-.18-.36-.26-.36-.36-.35-.46-.32-.59-.28-.73-.21-.88-.14-1.05L0 11.97l.06-1.22.16-1.04.24-.87.32-.71.36-.57.4-.44.42-.33.42-.24.4-.16.36-.1.32-.05.24-.01h.16l.06.01h8.16v-.83H6.24l-.01-2.75-.02-.37.05-.34.11-.31.17-.28.25-.26.31-.23.38-.2.44-.18.51-.15.58-.12.64-.1.71-.06.77-.04.84-.02 1.27.05 1.07.13zm-6.3 1.98l-.23.33-.08.41.08.41.23.34.33.22.41.09.41-.09.33-.22.23-.34.08-.41-.08-.41-.23-.33-.33-.22-.41-.09-.41.09-.33.22zM21.1 6.11l.28.06.32.12.35.18.36.27.36.35.35.47.32.59.28.73.21.88.14 1.04.05 1.23-.06 1.23-.16 1.04-.24.86-.32.71-.36.57-.4.45-.42.33-.42.24-.4.16-.36.09-.32.05-.24.02-.16-.01h-8.22v.82h5.84l.01 2.76.02.36-.05.34-.11.31-.17.29-.25.25-.31.24-.38.2-.44.17-.51.15-.58.13-.64.09-.71.07-.77.04-.84.01-1.27-.04-1.07-.14-.9-.2-.73-.25-.59-.3-.45-.33-.34-.34-.25-.34-.16-.33-.1-.3-.04-.25-.02-.2.01-.13v-5.34l.05-.64.13-.54.21-.46.26-.38.3-.32.33-.24.35-.2.35-.14.33-.1.3-.06.26-.04.21-.02.13-.01h5.84l.69-.05.59-.14.5-.21.41-.28.33-.32.27-.35.2-.36.15-.36.1-.35.07-.32.04-.28.02-.21V6.07h2.09l.14.01.21.03zm-6.47 14.25l-.23.33-.08.41.08.41.23.33.33.23.41.08.41-.08.33-.23.23-.33.08-.41-.08-.41-.23-.33-.33-.23-.41-.08-.41.08-.33.23z"/>"#;
// Dashboard label is translated, registry names stay as-is // Dashboard label is translated, registry names stay as-is
@@ -109,20 +109,6 @@ fn sidebar_dark(active_page: Option<&str>, t: &Translations) -> String {
("npm", "/ui/npm", "npm", npm_icon, false), ("npm", "/ui/npm", "npm", npm_icon, false),
("cargo", "/ui/cargo", "Cargo", cargo_icon, false), ("cargo", "/ui/cargo", "Cargo", cargo_icon, false),
("pypi", "/ui/pypi", "PyPI", pypi_icon, false), ("pypi", "/ui/pypi", "PyPI", pypi_icon, false),
(
"raw",
"/ui/raw",
"Raw",
r#"<path fill="currentColor" d="M14 2H6a2 2 0 00-2 2v16a2 2 0 002 2h12a2 2 0 002-2V8l-6-6zm4 18H6V4h7v5h5v11z"/>"#,
false,
),
(
"go",
"/ui/go",
"Go",
r#"<path fill="currentColor" d="M2.64 9.56s.24-.14.65-.38c.41-.24.97-.5 1.63-.7A7.85 7.85 0 017.53 8c.86 0 1.67.17 2.37.52.7.35 1.26.87 1.63 1.51.37.64.54 1.41.54 2.27v.2h-2.7v-.16c0-.47-.09-.86-.28-1.15a1.7 1.7 0 00-.77-.67 2.7 2.7 0 00-1.14-.22c-.56 0-1.06.13-1.46.4-.41.27-.72.66-.93 1.16-.21.5-.31 1.1-.31 1.8 0 .69.1 1.28.32 1.78.21.5.53.88.94 1.15.41.27.9.4 1.47.4.38 0 .73-.06 1.04-.17.31-.12.56-.29.74-.52.19-.23.29-.51.29-.84v-.14H7.15v-1.76h5.07v1.3c0 .8-.17 1.48-.52 2.04a3.46 3.46 0 01-1.5 1.3c-.66.3-1.44.45-2.35.45-.99 0-1.87-.18-2.63-.55a4.2 4.2 0 01-1.77-1.59C3.15 14.82 3 13.94 3 12.89v-.28c0-1.04.16-1.93.48-2.65a3.08 3.08 0 01-.84-.4zm12.1-1.34c.92 0 1.74.18 2.44.55a3.96 3.96 0 011.66 1.59c.4.7.6 1.54.6 2.53v.28c0 .99-.2 1.83-.6 2.53a3.96 3.96 0 01-1.66 1.59c-.7.37-1.52.55-2.44.55s-1.74-.18-2.44-.55a3.96 3.96 0 01-1.66-1.59c-.4-.7-.6-1.54-.6-2.53v-.28c0-.99.2-1.83.6-2.53a3.96 3.96 0 011.66-1.59c.7-.37 1.52-.55 2.44-.55zm0 2.12c-.44 0-.82.12-1.14.37-.32.24-.56.6-.73 1.06-.17.46-.26 1.01-.26 1.65v.28c0 .64.09 1.19.26 1.65.17.46.41.82.73 1.06.32.25.7.37 1.14.37.44 0 .82-.12 1.14-.37.32-.24.56-.6.73-1.06.17-.46.26-1.01.26-1.65v-.28c0-.64-.09-1.19-.26-1.65a2.17 2.17 0 00-.73-1.06 1.78 1.78 0 00-1.14-.37z"/>"#,
false,
),
]; ];
let nav_html: String = nav_items.iter().map(|(id, href, label, icon_path, is_stroke)| { let nav_html: String = nav_items.iter().map(|(id, href, label, icon_path, is_stroke)| {
@@ -291,15 +277,15 @@ pub fn render_registry_card(
) -> String { ) -> String {
format!( format!(
r##" r##"
<a href="{}" id="registry-{}" class="block bg-[#1e293b] rounded-lg border border-slate-700 p-3 hover:border-blue-400 transition-all"> <a href="{}" id="registry-{}" class="block bg-[#1e293b] rounded-lg border border-slate-700 p-4 md:p-6 hover:border-blue-400 transition-all">
<div class="flex items-center justify-between mb-2"> <div class="flex items-center justify-between mb-3">
<svg class="w-6 h-6 text-slate-400" fill="currentColor" viewBox="0 0 24 24"> <svg class="w-8 h-8 text-slate-400" fill="currentColor" viewBox="0 0 24 24">
{} {}
</svg> </svg>
<span class="text-[10px] font-medium text-green-400 bg-green-400/10 px-1.5 py-0.5 rounded-full">{}</span> <span class="text-xs font-medium text-green-400 bg-green-400/10 px-2 py-1 rounded-full">{}</span>
</div> </div>
<div class="text-sm font-semibold text-slate-200 mb-2">{}</div> <div class="text-lg font-semibold text-slate-200 mb-2">{}</div>
<div class="grid grid-cols-2 gap-1 text-xs"> <div class="grid grid-cols-2 gap-2 text-sm">
<div> <div>
<span class="text-slate-500">{}</span> <span class="text-slate-500">{}</span>
<div class="text-slate-300 font-medium">{}</div> <div class="text-slate-300 font-medium">{}</div>
@@ -347,9 +333,9 @@ pub fn render_mount_points_table(
format!( format!(
r##" r##"
<tr class="border-b border-slate-700"> <tr class="border-b border-slate-700">
<td class="px-4 py-3 text-slate-300">{}</td> <td class="py-3 text-slate-300">{}</td>
<td class="px-4 py-3 font-mono text-blue-400">{}</td> <td class="py-3 font-mono text-blue-400">{}</td>
<td class="px-4 py-3 text-slate-400">{}</td> <td class="py-3 text-slate-400">{}</td>
</tr> </tr>
"##, "##,
registry, mount_path, proxy_display registry, mount_path, proxy_display
@@ -372,7 +358,7 @@ pub fn render_mount_points_table(
<th class="px-4 py-2">{}</th> <th class="px-4 py-2">{}</th>
</tr> </tr>
</thead> </thead>
<tbody> <tbody class="px-4">
{} {}
</tbody> </tbody>
</table> </table>
@@ -402,11 +388,11 @@ pub fn render_activity_row(
format!( format!(
r##" r##"
<tr class="border-b border-slate-700/50 text-sm"> <tr class="border-b border-slate-700/50 text-sm">
<td class="px-4 py-2 text-slate-500">{}</td> <td class="py-2 text-slate-500">{}</td>
<td class="px-4 py-2 font-medium {}"><span class="px-2 py-0.5 bg-slate-700 rounded">{}</span></td> <td class="py-2 font-medium {}"><span class="px-2 py-0.5 bg-slate-700 rounded">{}</span></td>
<td class="px-4 py-2 text-slate-300 font-mono text-xs">{}</td> <td class="py-2 text-slate-300 font-mono text-xs">{}</td>
<td class="px-4 py-2 text-slate-400">{}</td> <td class="py-2 text-slate-400">{}</td>
<td class="px-4 py-2 text-slate-500">{}</td> <td class="py-2 text-slate-500">{}</td>
</tr> </tr>
"##, "##,
timestamp, timestamp,
@@ -438,7 +424,7 @@ pub fn render_activity_log(rows: &str, t: &Translations) -> String {
<th class="px-4 py-2">{}</th> <th class="px-4 py-2">{}</th>
</tr> </tr>
</thead> </thead>
<tbody> <tbody class="px-4">
{} {}
</tbody> </tbody>
</table> </table>
@@ -504,7 +490,7 @@ fn sidebar(active_page: Option<&str>) -> String {
let docker_icon = r#"<path fill="currentColor" d="M13.983 11.078h2.119a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.119a.185.185 0 00-.185.185v1.888c0 .102.083.185.185.185m-2.954-5.43h2.118a.186.186 0 00.186-.186V3.574a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.186m0 2.716h2.118a.187.187 0 00.186-.186V6.29a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.887c0 .102.082.185.185.186m-2.93 0h2.12a.186.186 0 00.184-.186V6.29a.185.185 0 00-.185-.185H8.1a.185.185 0 00-.185.185v1.887c0 .102.083.185.185.186m-2.964 0h2.119a.186.186 0 00.185-.186V6.29a.185.185 0 00-.185-.185H5.136a.186.186 0 00-.186.185v1.887c0 .102.084.185.186.186m5.893 2.715h2.118a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.185m-2.93 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.083.185.185.185m-2.964 0h2.119a.185.185 0 00.185-.185V9.006a.185.185 0 00-.185-.186h-2.12a.186.186 0 00-.185.186v1.887c0 .102.084.185.186.185m-2.92 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.082.185.185.185M23.763 9.89c-.065-.051-.672-.51-1.954-.51-.338.001-.676.03-1.01.087-.248-1.7-1.653-2.53-1.716-2.566l-.344-.199-.226.327c-.284.438-.49.922-.612 1.43-.23.97-.09 1.882.403 2.661-.595.332-1.55.413-1.744.42H.751a.751.751 0 00-.75.748 11.376 11.376 0 00.692 4.062c.545 1.428 1.355 2.48 2.41 3.124 1.18.723 3.1 1.137 5.275 1.137.983.003 1.963-.086 2.93-.266a12.248 12.248 0 003.823-1.389c.98-.567 1.86-1.288 2.61-2.136 1.252-1.418 1.998-2.997 2.553-4.4h.221c1.372 0 2.215-.549 2.68-1.009.309-.293.55-.65.707-1.046l.098-.288Z"/>"#; let docker_icon = r#"<path fill="currentColor" d="M13.983 11.078h2.119a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.119a.185.185 0 00-.185.185v1.888c0 .102.083.185.185.185m-2.954-5.43h2.118a.186.186 0 00.186-.186V3.574a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.186m0 2.716h2.118a.187.187 0 00.186-.186V6.29a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.887c0 .102.082.185.185.186m-2.93 0h2.12a.186.186 0 00.184-.186V6.29a.185.185 0 00-.185-.185H8.1a.185.185 0 00-.185.185v1.887c0 .102.083.185.185.186m-2.964 0h2.119a.186.186 0 00.185-.186V6.29a.185.185 0 00-.185-.185H5.136a.186.186 0 00-.186.185v1.887c0 .102.084.185.186.186m5.893 2.715h2.118a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.185m-2.93 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.083.185.185.185m-2.964 0h2.119a.185.185 0 00.185-.185V9.006a.185.185 0 00-.185-.186h-2.12a.186.186 0 00-.185.186v1.887c0 .102.084.185.186.185m-2.92 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.082.185.185.185M23.763 9.89c-.065-.051-.672-.51-1.954-.51-.338.001-.676.03-1.01.087-.248-1.7-1.653-2.53-1.716-2.566l-.344-.199-.226.327c-.284.438-.49.922-.612 1.43-.23.97-.09 1.882.403 2.661-.595.332-1.55.413-1.744.42H.751a.751.751 0 00-.75.748 11.376 11.376 0 00.692 4.062c.545 1.428 1.355 2.48 2.41 3.124 1.18.723 3.1 1.137 5.275 1.137.983.003 1.963-.086 2.93-.266a12.248 12.248 0 003.823-1.389c.98-.567 1.86-1.288 2.61-2.136 1.252-1.418 1.998-2.997 2.553-4.4h.221c1.372 0 2.215-.549 2.68-1.009.309-.293.55-.65.707-1.046l.098-.288Z"/>"#;
let maven_icon = r#"<path fill="currentColor" d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm-1 17.93c-3.95-.49-7-3.85-7-7.93 0-.62.08-1.21.21-1.79L9 15v1c0 1.1.9 2 2 2v1.93zm6.9-2.54c-.26-.81-1-1.39-1.9-1.39h-1v-3c0-.55-.45-1-1-1H8v-2h2c.55 0 1-.45 1-1V7h2c1.1 0 2-.9 2-2v-.41c2.93 1.19 5 4.06 5 7.41 0 2.08-.8 3.97-2.1 5.39z"/>"#; let maven_icon = r#"<path fill="currentColor" d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm-1 17.93c-3.95-.49-7-3.85-7-7.93 0-.62.08-1.21.21-1.79L9 15v1c0 1.1.9 2 2 2v1.93zm6.9-2.54c-.26-.81-1-1.39-1.9-1.39h-1v-3c0-.55-.45-1-1-1H8v-2h2c.55 0 1-.45 1-1V7h2c1.1 0 2-.9 2-2v-.41c2.93 1.19 5 4.06 5 7.41 0 2.08-.8 3.97-2.1 5.39z"/>"#;
let npm_icon = r#"<path fill="currentColor" d="M0 7.334v8h6.666v1.332H12v-1.332h12v-8H0zm6.666 6.664H5.334v-4H3.999v4H1.335V8.667h5.331v5.331zm4 0v1.336H8.001V8.667h5.334v5.332h-2.669v-.001zm12.001 0h-1.33v-4h-1.336v4h-1.335v-4h-1.33v4h-2.671V8.667h8.002v5.331zM10.665 10H12v2.667h-1.335V10z"/>"#; let npm_icon = r#"<path fill="currentColor" d="M0 7.334v8h6.666v1.332H12v-1.332h12v-8H0zm6.666 6.664H5.334v-4H3.999v4H1.335V8.667h5.331v5.331zm4 0v1.336H8.001V8.667h5.334v5.332h-2.669v-.001zm12.001 0h-1.33v-4h-1.336v4h-1.335v-4h-1.33v4h-2.671V8.667h8.002v5.331zM10.665 10H12v2.667h-1.335V10z"/>"#;
let cargo_icon = r#"<path fill="currentColor" d="M6 2h12a1 1 0 011 1v8a1 1 0 01-1 1H6a1 1 0 01-1-1V3a1 1 0 011-1zm0 2v2h12V4H6zm0 3v2h12V7H6zM2 14h8a1 1 0 011 1v6a1 1 0 01-1 1H2a1 1 0 01-1-1v-6a1 1 0 011-1zm0 2v1.5h8V16H2zM14 14h8a1 1 0 011 1v6a1 1 0 01-1 1h-8a1 1 0 01-1-1v-6a1 1 0 011-1zm0 2v1.5h8V16h-8z"/>"#; let cargo_icon = r#"<path fill="currentColor" d="M20 8h-3V4H3c-1.1 0-2 .9-2 2v11h2c0 1.66 1.34 3 3 3s3-1.34 3-3h6c0 1.66 1.34 3 3 3s3-1.34 3-3h2v-5l-3-4zM6 18.5c-.83 0-1.5-.67-1.5-1.5s.67-1.5 1.5-1.5 1.5.67 1.5 1.5-.67 1.5-1.5 1.5zm13.5-9l1.96 2.5H17V9.5h2.5zm-1.5 9c-.83 0-1.5-.67-1.5-1.5s.67-1.5 1.5-1.5 1.5.67 1.5 1.5-.67 1.5-1.5 1.5z"/>"#;
let pypi_icon = r#"<path fill="currentColor" d="M14.25.18l.9.2.73.26.59.3.45.32.34.34.25.34.16.33.1.3.04.26.02.2-.01.13V8.5l-.05.63-.13.55-.21.46-.26.38-.3.31-.33.25-.35.19-.35.14-.33.1-.3.07-.26.04-.21.02H8.83l-.69.05-.59.14-.5.22-.41.27-.33.32-.27.35-.2.36-.15.37-.1.35-.07.32-.04.27-.02.21v3.06H3.23l-.21-.03-.28-.07-.32-.12-.35-.18-.36-.26-.36-.36-.35-.46-.32-.59-.28-.73-.21-.88-.14-1.05L0 11.97l.06-1.22.16-1.04.24-.87.32-.71.36-.57.4-.44.42-.33.42-.24.4-.16.36-.1.32-.05.24-.01h.16l.06.01h8.16v-.83H6.24l-.01-2.75-.02-.37.05-.34.11-.31.17-.28.25-.26.31-.23.38-.2.44-.18.51-.15.58-.12.64-.1.71-.06.77-.04.84-.02 1.27.05 1.07.13zm-6.3 1.98l-.23.33-.08.41.08.41.23.34.33.22.41.09.41-.09.33-.22.23-.34.08-.41-.08-.41-.23-.33-.33-.22-.41-.09-.41.09-.33.22zM21.1 6.11l.28.06.32.12.35.18.36.27.36.35.35.47.32.59.28.73.21.88.14 1.04.05 1.23-.06 1.23-.16 1.04-.24.86-.32.71-.36.57-.4.45-.42.33-.42.24-.4.16-.36.09-.32.05-.24.02-.16-.01h-8.22v.82h5.84l.01 2.76.02.36-.05.34-.11.31-.17.29-.25.25-.31.24-.38.2-.44.17-.51.15-.58.13-.64.09-.71.07-.77.04-.84.01-1.27-.04-1.07-.14-.9-.2-.73-.25-.59-.3-.45-.33-.34-.34-.25-.34-.16-.33-.1-.3-.04-.25-.02-.2.01-.13v-5.34l.05-.64.13-.54.21-.46.26-.38.3-.32.33-.24.35-.2.35-.14.33-.1.3-.06.26-.04.21-.02.13-.01h5.84l.69-.05.59-.14.5-.21.41-.28.33-.32.27-.35.2-.36.15-.36.1-.35.07-.32.04-.28.02-.21V6.07h2.09l.14.01.21.03zm-6.47 14.25l-.23.33-.08.41.08.41.23.33.33.23.41.08.41-.08.33-.23.23-.33.08-.41-.08-.41-.23-.33-.33-.23-.41-.08-.41.08-.33.23z"/>"#; let pypi_icon = r#"<path fill="currentColor" d="M14.25.18l.9.2.73.26.59.3.45.32.34.34.25.34.16.33.1.3.04.26.02.2-.01.13V8.5l-.05.63-.13.55-.21.46-.26.38-.3.31-.33.25-.35.19-.35.14-.33.1-.3.07-.26.04-.21.02H8.83l-.69.05-.59.14-.5.22-.41.27-.33.32-.27.35-.2.36-.15.37-.1.35-.07.32-.04.27-.02.21v3.06H3.23l-.21-.03-.28-.07-.32-.12-.35-.18-.36-.26-.36-.36-.35-.46-.32-.59-.28-.73-.21-.88-.14-1.05L0 11.97l.06-1.22.16-1.04.24-.87.32-.71.36-.57.4-.44.42-.33.42-.24.4-.16.36-.1.32-.05.24-.01h.16l.06.01h8.16v-.83H6.24l-.01-2.75-.02-.37.05-.34.11-.31.17-.28.25-.26.31-.23.38-.2.44-.18.51-.15.58-.12.64-.1.71-.06.77-.04.84-.02 1.27.05 1.07.13zm-6.3 1.98l-.23.33-.08.41.08.41.23.34.33.22.41.09.41-.09.33-.22.23-.34.08-.41-.08-.41-.23-.33-.33-.22-.41-.09-.41.09-.33.22zM21.1 6.11l.28.06.32.12.35.18.36.27.36.35.35.47.32.59.28.73.21.88.14 1.04.05 1.23-.06 1.23-.16 1.04-.24.86-.32.71-.36.57-.4.45-.42.33-.42.24-.4.16-.36.09-.32.05-.24.02-.16-.01h-8.22v.82h5.84l.01 2.76.02.36-.05.34-.11.31-.17.29-.25.25-.31.24-.38.2-.44.17-.51.15-.58.13-.64.09-.71.07-.77.04-.84.01-1.27-.04-1.07-.14-.9-.2-.73-.25-.59-.3-.45-.33-.34-.34-.25-.34-.16-.33-.1-.3-.04-.25-.02-.2.01-.13v-5.34l.05-.64.13-.54.21-.46.26-.38.3-.32.33-.24.35-.2.35-.14.33-.1.3-.06.26-.04.21-.02.13-.01h5.84l.69-.05.59-.14.5-.21.41-.28.33-.32.27-.35.2-.36.15-.36.1-.35.07-.32.04-.28.02-.21V6.07h2.09l.14.01.21.03zm-6.47 14.25l-.23.33-.08.41.08.41.23.33.33.23.41.08.41-.08.33-.23.23-.33.08-.41-.08-.41-.23-.33-.33-.23-.41-.08-.41.08-.33.23z"/>"#;
let nav_items = [ let nav_items = [
@@ -520,20 +506,6 @@ fn sidebar(active_page: Option<&str>) -> String {
("npm", "/ui/npm", "npm", npm_icon, false), ("npm", "/ui/npm", "npm", npm_icon, false),
("cargo", "/ui/cargo", "Cargo", cargo_icon, false), ("cargo", "/ui/cargo", "Cargo", cargo_icon, false),
("pypi", "/ui/pypi", "PyPI", pypi_icon, false), ("pypi", "/ui/pypi", "PyPI", pypi_icon, false),
(
"raw",
"/ui/raw",
"Raw",
r#"<path fill="currentColor" d="M14 2H6a2 2 0 00-2 2v16a2 2 0 002 2h12a2 2 0 002-2V8l-6-6zm4 18H6V4h7v5h5v11z"/>"#,
false,
),
(
"go",
"/ui/go",
"Go",
r#"<path fill="currentColor" d="M2.64 9.56s.24-.14.65-.38c.41-.24.97-.5 1.63-.7A7.85 7.85 0 017.53 8c.86 0 1.67.17 2.37.52.7.35 1.26.87 1.63 1.51.37.64.54 1.41.54 2.27v.2h-2.7v-.16c0-.47-.09-.86-.28-1.15a1.7 1.7 0 00-.77-.67 2.7 2.7 0 00-1.14-.22c-.56 0-1.06.13-1.46.4-.41.27-.72.66-.93 1.16-.21.5-.31 1.1-.31 1.8 0 .69.1 1.28.32 1.78.21.5.53.88.94 1.15.41.27.9.4 1.47.4.38 0 .73-.06 1.04-.17.31-.12.56-.29.74-.52.19-.23.29-.51.29-.84v-.14H7.15v-1.76h5.07v1.3c0 .8-.17 1.48-.52 2.04a3.46 3.46 0 01-1.5 1.3c-.66.3-1.44.45-2.35.45-.99 0-1.87-.18-2.63-.55a4.2 4.2 0 01-1.77-1.59C3.15 14.82 3 13.94 3 12.89v-.28c0-1.04.16-1.93.48-2.65a3.08 3.08 0 01-.84-.4zm12.1-1.34c.92 0 1.74.18 2.44.55a3.96 3.96 0 011.66 1.59c.4.7.6 1.54.6 2.53v.28c0 .99-.2 1.83-.6 2.53a3.96 3.96 0 01-1.66 1.59c-.7.37-1.52.55-2.44.55s-1.74-.18-2.44-.55a3.96 3.96 0 01-1.66-1.59c-.4-.7-.6-1.54-.6-2.53v-.28c0-.99.2-1.83.6-2.53a3.96 3.96 0 011.66-1.59c.7-.37 1.52-.55 2.44-.55zm0 2.12c-.44 0-.82.12-1.14.37-.32.24-.56.6-.73 1.06-.17.46-.26 1.01-.26 1.65v.28c0 .64.09 1.19.26 1.65.17.46.41.82.73 1.06.32.25.7.37 1.14.37.44 0 .82-.12 1.14-.37.32-.24.56-.6.73-1.06.17-.46.26-1.01.26-1.65v-.28c0-.64-.09-1.19-.26-1.65a2.17 2.17 0 00-.73-1.06 1.78 1.78 0 00-1.14-.37z"/>"#,
false,
),
]; ];
let nav_html: String = nav_items.iter().map(|(id, href, label, icon_path, is_stroke)| { let nav_html: String = nav_items.iter().map(|(id, href, label, icon_path, is_stroke)| {
@@ -641,9 +613,7 @@ pub mod icons {
pub const DOCKER: &str = r#"<path fill="currentColor" d="M13.983 11.078h2.119a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.119a.185.185 0 00-.185.185v1.888c0 .102.083.185.185.185m-2.954-5.43h2.118a.186.186 0 00.186-.186V3.574a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.186m0 2.716h2.118a.187.187 0 00.186-.186V6.29a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.887c0 .102.082.185.185.186m-2.93 0h2.12a.186.186 0 00.184-.186V6.29a.185.185 0 00-.185-.185H8.1a.185.185 0 00-.185.185v1.887c0 .102.083.185.185.186m-2.964 0h2.119a.186.186 0 00.185-.186V6.29a.185.185 0 00-.185-.185H5.136a.186.186 0 00-.186.185v1.887c0 .102.084.185.186.186m5.893 2.715h2.118a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.185m-2.93 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.083.185.185.185m-2.964 0h2.119a.185.185 0 00.185-.185V9.006a.185.185 0 00-.185-.186h-2.12a.186.186 0 00-.185.186v1.887c0 .102.084.185.186.185m-2.92 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.082.185.185.185M23.763 9.89c-.065-.051-.672-.51-1.954-.51-.338.001-.676.03-1.01.087-.248-1.7-1.653-2.53-1.716-2.566l-.344-.199-.226.327c-.284.438-.49.922-.612 1.43-.23.97-.09 1.882.403 2.661-.595.332-1.55.413-1.744.42H.751a.751.751 0 00-.75.748 11.376 11.376 0 00.692 4.062c.545 1.428 1.355 2.48 2.41 3.124 1.18.723 3.1 1.137 5.275 1.137.983.003 1.963-.086 2.93-.266a12.248 12.248 0 003.823-1.389c.98-.567 1.86-1.288 2.61-2.136 1.252-1.418 1.998-2.997 2.553-4.4h.221c1.372 0 2.215-.549 2.68-1.009.309-.293.55-.65.707-1.046l.098-.288Z"/>"#; pub const DOCKER: &str = r#"<path fill="currentColor" d="M13.983 11.078h2.119a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.119a.185.185 0 00-.185.185v1.888c0 .102.083.185.185.185m-2.954-5.43h2.118a.186.186 0 00.186-.186V3.574a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.186m0 2.716h2.118a.187.187 0 00.186-.186V6.29a.186.186 0 00-.186-.185h-2.118a.185.185 0 00-.185.185v1.887c0 .102.082.185.185.186m-2.93 0h2.12a.186.186 0 00.184-.186V6.29a.185.185 0 00-.185-.185H8.1a.185.185 0 00-.185.185v1.887c0 .102.083.185.185.186m-2.964 0h2.119a.186.186 0 00.185-.186V6.29a.185.185 0 00-.185-.185H5.136a.186.186 0 00-.186.185v1.887c0 .102.084.185.186.186m5.893 2.715h2.118a.186.186 0 00.186-.185V9.006a.186.186 0 00-.186-.186h-2.118a.185.185 0 00-.185.185v1.888c0 .102.082.185.185.185m-2.93 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.083.185.185.185m-2.964 0h2.119a.185.185 0 00.185-.185V9.006a.185.185 0 00-.185-.186h-2.12a.186.186 0 00-.185.186v1.887c0 .102.084.185.186.185m-2.92 0h2.12a.185.185 0 00.184-.185V9.006a.185.185 0 00-.184-.186h-2.12a.185.185 0 00-.184.185v1.888c0 .102.082.185.185.185M23.763 9.89c-.065-.051-.672-.51-1.954-.51-.338.001-.676.03-1.01.087-.248-1.7-1.653-2.53-1.716-2.566l-.344-.199-.226.327c-.284.438-.49.922-.612 1.43-.23.97-.09 1.882.403 2.661-.595.332-1.55.413-1.744.42H.751a.751.751 0 00-.75.748 11.376 11.376 0 00.692 4.062c.545 1.428 1.355 2.48 2.41 3.124 1.18.723 3.1 1.137 5.275 1.137.983.003 1.963-.086 2.93-.266a12.248 12.248 0 003.823-1.389c.98-.567 1.86-1.288 2.61-2.136 1.252-1.418 1.998-2.997 2.553-4.4h.221c1.372 0 2.215-.549 2.68-1.009.309-.293.55-.65.707-1.046l.098-.288Z"/>"#;
pub const MAVEN: &str = r#"<path fill="currentColor" d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm-1 17.93c-3.95-.49-7-3.85-7-7.93 0-.62.08-1.21.21-1.79L9 15v1c0 1.1.9 2 2 2v1.93zm6.9-2.54c-.26-.81-1-1.39-1.9-1.39h-1v-3c0-.55-.45-1-1-1H8v-2h2c.55 0 1-.45 1-1V7h2c1.1 0 2-.9 2-2v-.41c2.93 1.19 5 4.06 5 7.41 0 2.08-.8 3.97-2.1 5.39z"/>"#; pub const MAVEN: &str = r#"<path fill="currentColor" d="M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm-1 17.93c-3.95-.49-7-3.85-7-7.93 0-.62.08-1.21.21-1.79L9 15v1c0 1.1.9 2 2 2v1.93zm6.9-2.54c-.26-.81-1-1.39-1.9-1.39h-1v-3c0-.55-.45-1-1-1H8v-2h2c.55 0 1-.45 1-1V7h2c1.1 0 2-.9 2-2v-.41c2.93 1.19 5 4.06 5 7.41 0 2.08-.8 3.97-2.1 5.39z"/>"#;
pub const NPM: &str = r#"<path fill="currentColor" d="M0 7.334v8h6.666v1.332H12v-1.332h12v-8H0zm6.666 6.664H5.334v-4H3.999v4H1.335V8.667h5.331v5.331zm4 0v1.336H8.001V8.667h5.334v5.332h-2.669v-.001zm12.001 0h-1.33v-4h-1.336v4h-1.335v-4h-1.33v4h-2.671V8.667h8.002v5.331zM10.665 10H12v2.667h-1.335V10z"/>"#; pub const NPM: &str = r#"<path fill="currentColor" d="M0 7.334v8h6.666v1.332H12v-1.332h12v-8H0zm6.666 6.664H5.334v-4H3.999v4H1.335V8.667h5.331v5.331zm4 0v1.336H8.001V8.667h5.334v5.332h-2.669v-.001zm12.001 0h-1.33v-4h-1.336v4h-1.335v-4h-1.33v4h-2.671V8.667h8.002v5.331zM10.665 10H12v2.667h-1.335V10z"/>"#;
pub const CARGO: &str = r#"<path fill="currentColor" d="M6 2h12a1 1 0 011 1v8a1 1 0 01-1 1H6a1 1 0 01-1-1V3a1 1 0 011-1zm0 2v2h12V4H6zm0 3v2h12V7H6zM2 14h8a1 1 0 011 1v6a1 1 0 01-1 1H2a1 1 0 01-1-1v-6a1 1 0 011-1zm0 2v1.5h8V16H2zM14 14h8a1 1 0 011 1v6a1 1 0 01-1 1h-8a1 1 0 01-1-1v-6a1 1 0 011-1zm0 2v1.5h8V16h-8z"/>"#; pub const CARGO: &str = r#"<path fill="currentColor" d="M20 8h-3V4H3c-1.1 0-2 .9-2 2v11h2c0 1.66 1.34 3 3 3s3-1.34 3-3h6c0 1.66 1.34 3 3 3s3-1.34 3-3h2v-5l-3-4zM6 18.5c-.83 0-1.5-.67-1.5-1.5s.67-1.5 1.5-1.5 1.5.67 1.5 1.5-.67 1.5-1.5 1.5zm13.5-9l1.96 2.5H17V9.5h2.5zm-1.5 9c-.83 0-1.5-.67-1.5-1.5s.67-1.5 1.5-1.5 1.5.67 1.5 1.5-.67 1.5-1.5 1.5z"/>"#;
pub const GO: &str = r#"<path fill="currentColor" d="M2.64 9.56s.24-.14.65-.38c.41-.24.97-.5 1.63-.7A7.85 7.85 0 017.53 8c.86 0 1.67.17 2.37.52.7.35 1.26.87 1.63 1.51.37.64.54 1.41.54 2.27v.2h-2.7v-.16c0-.47-.09-.86-.28-1.15a1.7 1.7 0 00-.77-.67 2.7 2.7 0 00-1.14-.22c-.56 0-1.06.13-1.46.4-.41.27-.72.66-.93 1.16-.21.5-.31 1.1-.31 1.8 0 .69.1 1.28.32 1.78.21.5.53.88.94 1.15.41.27.9.4 1.47.4.38 0 .73-.06 1.04-.17.31-.12.56-.29.74-.52.19-.23.29-.51.29-.84v-.14H7.15v-1.76h5.07v1.3c0 .8-.17 1.48-.52 2.04a3.46 3.46 0 01-1.5 1.3c-.66.3-1.44.45-2.35.45-.99 0-1.87-.18-2.63-.55a4.2 4.2 0 01-1.77-1.59C3.15 14.82 3 13.94 3 12.89v-.28c0-1.04.16-1.93.48-2.65a3.08 3.08 0 01-.84-.4zm12.1-1.34c.92 0 1.74.18 2.44.55a3.96 3.96 0 011.66 1.59c.4.7.6 1.54.6 2.53v.28c0 .99-.2 1.83-.6 2.53a3.96 3.96 0 01-1.66 1.59c-.7.37-1.52.55-2.44.55s-1.74-.18-2.44-.55a3.96 3.96 0 01-1.66-1.59c-.4-.7-.6-1.54-.6-2.53v-.28c0-.99.2-1.83.6-2.53a3.96 3.96 0 011.66-1.59c.7-.37 1.52-.55 2.44-.55zm0 2.12c-.44 0-.82.12-1.14.37-.32.24-.56.6-.73 1.06-.17.46-.26 1.01-.26 1.65v.28c0 .64.09 1.19.26 1.65.17.46.41.82.73 1.06.32.25.7.37 1.14.37.44 0 .82-.12 1.14-.37.32-.24.56-.6.73-1.06.17-.46.26-1.01.26-1.65v-.28c0-.64-.09-1.19-.26-1.65a2.17 2.17 0 00-.73-1.06 1.78 1.78 0 00-1.14-.37z"/>"#;
pub const RAW: &str = r#"<path fill="currentColor" d="M14 2H6a2 2 0 00-2 2v16a2 2 0 002 2h12a2 2 0 002-2V8l-6-6zm4 18H6V4h7v5h5v11z"/>"#;
pub const PYPI: &str = r#"<path fill="currentColor" d="M14.25.18l.9.2.73.26.59.3.45.32.34.34.25.34.16.33.1.3.04.26.02.2-.01.13V8.5l-.05.63-.13.55-.21.46-.26.38-.3.31-.33.25-.35.19-.35.14-.33.1-.3.07-.26.04-.21.02H8.83l-.69.05-.59.14-.5.22-.41.27-.33.32-.27.35-.2.36-.15.37-.1.35-.07.32-.04.27-.02.21v3.06H3.23l-.21-.03-.28-.07-.32-.12-.35-.18-.36-.26-.36-.36-.35-.46-.32-.59-.28-.73-.21-.88-.14-1.05L0 11.97l.06-1.22.16-1.04.24-.87.32-.71.36-.57.4-.44.42-.33.42-.24.4-.16.36-.1.32-.05.24-.01h.16l.06.01h8.16v-.83H6.24l-.01-2.75-.02-.37.05-.34.11-.31.17-.28.25-.26.31-.23.38-.2.44-.18.51-.15.58-.12.64-.1.71-.06.77-.04.84-.02 1.27.05 1.07.13zm-6.3 1.98l-.23.33-.08.41.08.41.23.34.33.22.41.09.41-.09.33-.22.23-.34.08-.41-.08-.41-.23-.33-.33-.22-.41-.09-.41.09-.33.22zM21.1 6.11l.28.06.32.12.35.18.36.27.36.35.35.47.32.59.28.73.21.88.14 1.04.05 1.23-.06 1.23-.16 1.04-.24.86-.32.71-.36.57-.4.45-.42.33-.42.24-.4.16-.36.09-.32.05-.24.02-.16-.01h-8.22v.82h5.84l.01 2.76.02.36-.05.34-.11.31-.17.29-.25.25-.31.24-.38.2-.44.17-.51.15-.58.13-.64.09-.71.07-.77.04-.84.01-1.27-.04-1.07-.14-.9-.2-.73-.25-.59-.3-.45-.33-.34-.34-.25-.34-.16-.33-.1-.3-.04-.25-.02-.2.01-.13v-5.34l.05-.64.13-.54.21-.46.26-.38.3-.32.33-.24.35-.2.35-.14.33-.1.3-.06.26-.04.21-.02.13-.01h5.84l.69-.05.59-.14.5-.21.41-.28.33-.32.27-.35.2-.36.15-.36.1-.35.07-.32.04-.28.02-.21V6.07h2.09l.14.01.21.03zm-6.47 14.25l-.23.33-.08.41.08.41.23.33.33.23.41.08.41-.08.33-.23.23-.33.08-.41-.08-.41-.23-.33-.33-.23-.41-.08-.41.08-.33.23z"/>"#; pub const PYPI: &str = r#"<path fill="currentColor" d="M14.25.18l.9.2.73.26.59.3.45.32.34.34.25.34.16.33.1.3.04.26.02.2-.01.13V8.5l-.05.63-.13.55-.21.46-.26.38-.3.31-.33.25-.35.19-.35.14-.33.1-.3.07-.26.04-.21.02H8.83l-.69.05-.59.14-.5.22-.41.27-.33.32-.27.35-.2.36-.15.37-.1.35-.07.32-.04.27-.02.21v3.06H3.23l-.21-.03-.28-.07-.32-.12-.35-.18-.36-.26-.36-.36-.35-.46-.32-.59-.28-.73-.21-.88-.14-1.05L0 11.97l.06-1.22.16-1.04.24-.87.32-.71.36-.57.4-.44.42-.33.42-.24.4-.16.36-.1.32-.05.24-.01h.16l.06.01h8.16v-.83H6.24l-.01-2.75-.02-.37.05-.34.11-.31.17-.28.25-.26.31-.23.38-.2.44-.18.51-.15.58-.12.64-.1.71-.06.77-.04.84-.02 1.27.05 1.07.13zm-6.3 1.98l-.23.33-.08.41.08.41.23.34.33.22.41.09.41-.09.33-.22.23-.34.08-.41-.08-.41-.23-.33-.33-.22-.41-.09-.41.09-.33.22zM21.1 6.11l.28.06.32.12.35.18.36.27.36.35.35.47.32.59.28.73.21.88.14 1.04.05 1.23-.06 1.23-.16 1.04-.24.86-.32.71-.36.57-.4.45-.42.33-.42.24-.4.16-.36.09-.32.05-.24.02-.16-.01h-8.22v.82h5.84l.01 2.76.02.36-.05.34-.11.31-.17.29-.25.25-.31.24-.38.2-.44.17-.51.15-.58.13-.64.09-.71.07-.77.04-.84.01-1.27-.04-1.07-.14-.9-.2-.73-.25-.59-.3-.45-.33-.34-.34-.25-.34-.16-.33-.1-.3-.04-.25-.02-.2.01-.13v-5.34l.05-.64.13-.54.21-.46.26-.38.3-.32.33-.24.35-.2.35-.14.33-.1.3-.06.26-.04.21-.02.13-.01h5.84l.69-.05.59-.14.5-.21.41-.28.33-.32.27-.35.2-.36.15-.36.1-.35.07-.32.04-.28.02-.21V6.07h2.09l.14.01.21.03zm-6.47 14.25l-.23.33-.08.41.08.41.23.33.33.23.41.08.41-.08.33-.23.23-.33.08-.41-.08-.41-.23-.33-.33-.23-.41-.08-.41.08-.33.23z"/>"#;
} }
@@ -705,7 +675,7 @@ pub fn render_bragging_footer(lang: Lang) -> String {
</div> </div>
<div class="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-6 gap-4 text-center"> <div class="grid grid-cols-2 md:grid-cols-3 lg:grid-cols-6 gap-4 text-center">
<div class="p-3"> <div class="p-3">
<div class="text-2xl font-bold text-blue-400">32 MB</div> <div class="text-2xl font-bold text-blue-400">34 MB</div>
<div class="text-xs text-slate-500 mt-1">{}</div> <div class="text-xs text-slate-500 mt-1">{}</div>
</div> </div>
<div class="p-3"> <div class="p-3">
@@ -717,7 +687,7 @@ pub fn render_bragging_footer(lang: Lang) -> String {
<div class="text-xs text-slate-500 mt-1">{}</div> <div class="text-xs text-slate-500 mt-1">{}</div>
</div> </div>
<div class="p-3"> <div class="p-3">
<div class="text-2xl font-bold text-yellow-400">7</div> <div class="text-2xl font-bold text-yellow-400">5</div>
<div class="text-xs text-slate-500 mt-1">{}</div> <div class="text-xs text-slate-500 mt-1">{}</div>
</div> </div>
<div class="p-3"> <div class="p-3">

View File

@@ -87,10 +87,6 @@ pub fn routes() -> Router<Arc<AppState>> {
.route("/ui/cargo/{name}", get(cargo_detail)) .route("/ui/cargo/{name}", get(cargo_detail))
.route("/ui/pypi", get(pypi_list)) .route("/ui/pypi", get(pypi_list))
.route("/ui/pypi/{name}", get(pypi_detail)) .route("/ui/pypi/{name}", get(pypi_detail))
.route("/ui/go", get(go_list))
.route("/ui/go/{*name}", get(go_detail))
.route("/ui/raw", get(raw_list))
.route("/ui/raw/{*name}", get(raw_detail))
// API endpoints for HTMX // API endpoints for HTMX
.route("/api/ui/stats", get(api_stats)) .route("/api/ui/stats", get(api_stats))
.route("/api/ui/dashboard", get(api_dashboard)) .route("/api/ui/dashboard", get(api_dashboard))
@@ -302,79 +298,3 @@ async fn pypi_detail(
let detail = get_pypi_detail(&state.storage, &name).await; let detail = get_pypi_detail(&state.storage, &name).await;
Html(render_package_detail("pypi", &name, &detail, lang)) Html(render_package_detail("pypi", &name, &detail, lang))
} }
// Go pages
async fn go_list(
State(state): State<Arc<AppState>>,
Query(query): Query<ListQuery>,
headers: axum::http::HeaderMap,
) -> impl IntoResponse {
let lang = extract_lang_from_list(&query, headers.get("cookie").and_then(|v| v.to_str().ok()));
let page = query.page.unwrap_or(1).max(1);
let limit = query.limit.unwrap_or(DEFAULT_PAGE_SIZE).min(100);
let all_modules = state.repo_index.get("go", &state.storage).await;
let (modules, total) = paginate(&all_modules, page, limit);
Html(render_registry_list_paginated(
"go",
"Go Modules",
&modules,
page,
limit,
total,
lang,
))
}
async fn go_detail(
State(state): State<Arc<AppState>>,
Path(name): Path<String>,
Query(query): Query<LangQuery>,
headers: axum::http::HeaderMap,
) -> impl IntoResponse {
let lang = extract_lang(
&Query(query),
headers.get("cookie").and_then(|v| v.to_str().ok()),
);
let detail = get_go_detail(&state.storage, &name).await;
Html(render_package_detail("go", &name, &detail, lang))
}
// Raw pages
async fn raw_list(
State(state): State<Arc<AppState>>,
Query(query): Query<ListQuery>,
headers: axum::http::HeaderMap,
) -> impl IntoResponse {
let lang = extract_lang_from_list(&query, headers.get("cookie").and_then(|v| v.to_str().ok()));
let page = query.page.unwrap_or(1).max(1);
let limit = query.limit.unwrap_or(DEFAULT_PAGE_SIZE).min(100);
let all_files = state.repo_index.get("raw", &state.storage).await;
let (files, total) = paginate(&all_files, page, limit);
Html(render_registry_list_paginated(
"raw",
"Raw Storage",
&files,
page,
limit,
total,
lang,
))
}
async fn raw_detail(
State(state): State<Arc<AppState>>,
Path(name): Path<String>,
Query(query): Query<LangQuery>,
headers: axum::http::HeaderMap,
) -> impl IntoResponse {
let lang = extract_lang(
&Query(query),
headers.get("cookie").and_then(|v| v.to_str().ok()),
);
let detail = get_raw_detail(&state.storage, &name).await;
Html(render_package_detail("raw", &name, &detail, lang))
}

View File

@@ -24,8 +24,22 @@ pub fn render_dashboard(data: &DashboardResponse, lang: Lang) -> String {
.registry_stats .registry_stats
.iter() .iter()
.map(|r| { .map(|r| {
let icon = get_registry_icon(&r.name); let icon = match r.name.as_str() {
let display_name = get_registry_title(&r.name); "docker" => icons::DOCKER,
"maven" => icons::MAVEN,
"npm" => icons::NPM,
"cargo" => icons::CARGO,
"pypi" => icons::PYPI,
_ => icons::DOCKER,
};
let display_name = match r.name.as_str() {
"docker" => "Docker",
"maven" => "Maven",
"npm" => "npm",
"cargo" => "Cargo",
"pypi" => "PyPI",
_ => &r.name,
};
render_registry_card( render_registry_card(
display_name, display_name,
icon, icon,
@@ -60,56 +74,16 @@ pub fn render_dashboard(data: &DashboardResponse, lang: Lang) -> String {
t.no_activity t.no_activity
) )
} else { } else {
// Group consecutive identical entries (same action+artifact+registry+source) data.activity
struct GroupedActivity {
time: String,
action: String,
artifact: String,
registry: String,
source: String,
count: usize,
}
let mut grouped: Vec<GroupedActivity> = Vec::new();
for entry in &data.activity {
let action = entry.action.to_string();
let is_repeat = grouped.last().is_some_and(|last| {
last.action == action
&& last.artifact == entry.artifact
&& last.registry == entry.registry
&& last.source == entry.source
});
if is_repeat {
if let Some(last) = grouped.last_mut() {
last.count += 1;
}
} else {
grouped.push(GroupedActivity {
time: format_relative_time(&entry.timestamp),
action,
artifact: entry.artifact.clone(),
registry: entry.registry.clone(),
source: entry.source.clone(),
count: 1,
});
}
}
grouped
.iter() .iter()
.map(|g| { .map(|entry| {
let display_artifact = if g.count > 1 { let time_ago = format_relative_time(&entry.timestamp);
format!("{} (x{})", g.artifact, g.count)
} else {
g.artifact.clone()
};
render_activity_row( render_activity_row(
&g.time, &time_ago,
&g.action, &entry.action.to_string(),
&display_artifact, &entry.artifact,
&g.registry, &entry.registry,
&g.source, &entry.source,
) )
}) })
.collect() .collect()
@@ -141,7 +115,7 @@ pub fn render_dashboard(data: &DashboardResponse, lang: Lang) -> String {
{} {}
<div class="grid grid-cols-2 md:grid-cols-4 lg:grid-cols-7 gap-3 mb-6"> <div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 xl:grid-cols-5 gap-4 mb-6">
{} {}
</div> </div>
@@ -655,8 +629,6 @@ pub fn render_package_detail(
"pip install {} --index-url http://127.0.0.1:4000/simple", "pip install {} --index-url http://127.0.0.1:4000/simple",
name name
), ),
"go" => format!("GOPROXY=http://127.0.0.1:4000/go go get {}", name),
"raw" => format!("curl -O http://127.0.0.1:4000/raw/{}/<file>", name),
_ => String::new(), _ => String::new(),
}; };
@@ -823,8 +795,6 @@ fn get_registry_icon(registry_type: &str) -> &'static str {
"npm" => icons::NPM, "npm" => icons::NPM,
"cargo" => icons::CARGO, "cargo" => icons::CARGO,
"pypi" => icons::PYPI, "pypi" => icons::PYPI,
"go" => icons::GO,
"raw" => icons::RAW,
_ => { _ => {
r#"<path fill="currentColor" d="M10 4H4c-1.1 0-1.99.9-1.99 2L2 18c0 1.1.9 2 2 2h16c1.1 0 2-.9 2-2V8c0-1.1-.9-2-2-2h-8l-2-2z"/>"# r#"<path fill="currentColor" d="M10 4H4c-1.1 0-1.99.9-1.99 2L2 18c0 1.1.9 2 2 2h16c1.1 0 2-.9 2-2V8c0-1.1-.9-2-2-2h-8l-2-2z"/>"#
} }
@@ -838,8 +808,6 @@ fn get_registry_title(registry_type: &str) -> &'static str {
"npm" => "npm Registry", "npm" => "npm Registry",
"cargo" => "Cargo Registry", "cargo" => "Cargo Registry",
"pypi" => "PyPI Repository", "pypi" => "PyPI Repository",
"go" => "Go Modules",
"raw" => "Raw Storage",
_ => "Registry", _ => "Registry",
} }
} }

View File

@@ -178,12 +178,7 @@ pub fn validate_docker_name(name: &str) -> Result<(), ValidationError> {
"empty path segment".to_string(), "empty path segment".to_string(),
)); ));
} }
// Safety: segment.is_empty() checked above, but use match for defense-in-depth let first = segment.chars().next().unwrap();
let Some(first) = segment.chars().next() else {
return Err(ValidationError::InvalidDockerName(
"empty path segment".to_string(),
));
};
if !first.is_ascii_alphanumeric() { if !first.is_ascii_alphanumeric() {
return Err(ValidationError::InvalidDockerName( return Err(ValidationError::InvalidDockerName(
"segment must start with alphanumeric".to_string(), "segment must start with alphanumeric".to_string(),
@@ -297,10 +292,7 @@ pub fn validate_docker_reference(reference: &str) -> Result<(), ValidationError>
} }
// Validate as tag // Validate as tag
// Safety: empty check at function start, but use let-else for defense-in-depth let first = reference.chars().next().unwrap();
let Some(first) = reference.chars().next() else {
return Err(ValidationError::EmptyInput);
};
if !first.is_ascii_alphanumeric() { if !first.is_ascii_alphanumeric() {
return Err(ValidationError::InvalidReference( return Err(ValidationError::InvalidReference(
"tag must start with alphanumeric".to_string(), "tag must start with alphanumeric".to_string(),
@@ -504,150 +496,3 @@ mod tests {
assert!(validate_docker_reference("-dash").is_err()); assert!(validate_docker_reference("-dash").is_err());
} }
} }
#[cfg(test)]
mod proptests {
use super::*;
use proptest::prelude::*;
/// Valid lowercase Docker name component
fn docker_component() -> impl Strategy<Value = String> {
"[a-z0-9][a-z0-9._-]{0,30}".prop_filter("no consecutive separators", |s| {
!s.contains("..") && !s.contains("//") && !s.contains("--") && !s.contains("__")
})
}
/// Valid sha256 hex string
fn sha256_hex() -> impl Strategy<Value = String> {
"[0-9a-f]{64}"
}
/// Valid Docker tag (no `..` or `/` which trigger path traversal rejection)
fn docker_tag() -> impl Strategy<Value = String> {
"[a-zA-Z0-9][a-zA-Z0-9._-]{0,50}".prop_filter("no path traversal", |s| {
!s.contains("..") && !s.contains('/')
})
}
// === validate_storage_key ===
proptest! {
#[test]
fn storage_key_never_panics(s in "\\PC{0,2000}") {
let _ = validate_storage_key(&s);
}
#[test]
fn storage_key_rejects_path_traversal(
prefix in "[a-z]{0,10}",
suffix in "[a-z]{0,10}"
) {
let key = format!("{}/../{}", prefix, suffix);
prop_assert!(validate_storage_key(&key).is_err());
}
#[test]
fn storage_key_rejects_absolute(path in "/[a-z/]{1,50}") {
prop_assert!(validate_storage_key(&path).is_err());
}
#[test]
fn storage_key_accepts_valid(
segments in prop::collection::vec("[a-z0-9]{1,20}", 1..5)
) {
let key = segments.join("/");
prop_assert!(validate_storage_key(&key).is_ok());
}
}
// === validate_docker_name ===
proptest! {
#[test]
fn docker_name_never_panics(s in "\\PC{0,500}") {
let _ = validate_docker_name(&s);
}
#[test]
fn docker_name_accepts_valid_single(name in docker_component()) {
prop_assert!(validate_docker_name(&name).is_ok());
}
#[test]
fn docker_name_accepts_valid_path(
components in prop::collection::vec(docker_component(), 1..4)
) {
let name = components.join("/");
prop_assert!(validate_docker_name(&name).is_ok());
}
#[test]
fn docker_name_rejects_uppercase(
lower in "[a-z]{1,10}",
upper in "[A-Z]{1,10}"
) {
let name = format!("{}{}", lower, upper);
prop_assert!(validate_docker_name(&name).is_err());
}
}
// === validate_digest ===
proptest! {
#[test]
fn digest_never_panics(s in "\\PC{0,200}") {
let _ = validate_digest(&s);
}
#[test]
fn digest_sha256_roundtrip(hash in sha256_hex()) {
let digest = format!("sha256:{}", hash);
prop_assert!(validate_digest(&digest).is_ok());
}
#[test]
fn digest_sha512_roundtrip(hash in "[0-9a-f]{128}") {
let digest = format!("sha512:{}", hash);
prop_assert!(validate_digest(&digest).is_ok());
}
#[test]
fn digest_wrong_algo_rejected(
algo in "[a-z]{2,8}",
hash in "[0-9a-f]{64}"
) {
prop_assume!(algo != "sha256" && algo != "sha512");
let digest = format!("{}:{}", algo, hash);
prop_assert!(validate_digest(&digest).is_err());
}
}
// === validate_docker_reference ===
proptest! {
#[test]
fn reference_never_panics(s in "\\PC{0,200}") {
let _ = validate_docker_reference(&s);
}
#[test]
fn reference_accepts_valid_tag(tag in docker_tag()) {
prop_assert!(validate_docker_reference(&tag).is_ok());
}
#[test]
fn reference_accepts_valid_digest(hash in sha256_hex()) {
let reference = format!("sha256:{}", hash);
prop_assert!(validate_docker_reference(&reference).is_ok());
}
#[test]
fn reference_rejects_traversal(
prefix in "[a-z]{0,5}",
suffix in "[a-z]{0,5}"
) {
let reference = format!("{}../{}", prefix, suffix);
prop_assert!(validate_docker_reference(&reference).is_err());
}
}
}

28
nora-storage/Cargo.toml Normal file
View File

@@ -0,0 +1,28 @@
[package]
name = "nora-storage"
version.workspace = true
edition.workspace = true
license.workspace = true
authors.workspace = true
repository.workspace = true
homepage.workspace = true
description = "S3-compatible storage server for NORA"
[[bin]]
name = "nora-storage"
path = "src/main.rs"
[dependencies]
tokio.workspace = true
axum.workspace = true
serde.workspace = true
serde_json.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
toml = "1.0"
uuid = { version = "1", features = ["v4"] }
sha2 = "0.10"
base64 = "0.22"
httpdate = "1"
chrono = { version = "0.4", features = ["serde"] }
quick-xml = { version = "0.39", features = ["serialize"] }

View File

@@ -0,0 +1,47 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT
use serde::{Deserialize, Serialize};
use std::fs;
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Config {
pub server: ServerConfig,
pub storage: StorageConfig,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ServerConfig {
pub host: String,
pub port: u16,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct StorageConfig {
pub data_dir: String,
pub max_body_size: usize,
}
impl Config {
pub fn load() -> Self {
fs::read_to_string("config.toml")
.ok()
.and_then(|content| toml::from_str(&content).ok())
.unwrap_or_default()
}
}
impl Default for Config {
fn default() -> Self {
Self {
server: ServerConfig {
host: String::from("127.0.0.1"),
port: 3000,
},
storage: StorageConfig {
data_dir: String::from("data"),
max_body_size: 1024 * 1024 * 1024, // 1GB
},
}
}
}

311
nora-storage/src/main.rs Normal file
View File

@@ -0,0 +1,311 @@
// Copyright (c) 2026 Volkov Pavel | DevITWay
// SPDX-License-Identifier: MIT
mod config;
use axum::extract::DefaultBodyLimit;
use axum::{
body::Bytes,
extract::{Path, State},
http::StatusCode,
response::{IntoResponse, Response},
routing::{delete, get, put},
Router,
};
use chrono::Utc;
use config::Config;
use quick_xml::se::to_string as to_xml;
use serde::Serialize;
use std::fs;
use std::sync::Arc;
use tracing::info;
pub struct AppState {
pub config: Config,
}
#[derive(Serialize)]
#[serde(rename = "ListAllMyBucketsResult")]
struct ListBucketsResult {
#[serde(rename = "Buckets")]
buckets: Buckets,
}
#[derive(Serialize)]
struct Buckets {
#[serde(rename = "Bucket")]
bucket: Vec<BucketInfo>,
}
#[derive(Serialize)]
struct BucketInfo {
#[serde(rename = "Name")]
name: String,
#[serde(rename = "CreationDate")]
creation_date: String,
}
#[derive(Serialize)]
#[serde(rename = "ListBucketResult")]
struct ListObjectsResult {
#[serde(rename = "Name")]
name: String,
#[serde(rename = "Contents")]
contents: Vec<ObjectInfo>,
}
#[derive(Serialize)]
struct ObjectInfo {
#[serde(rename = "Key")]
key: String,
#[serde(rename = "Size")]
size: u64,
#[serde(rename = "LastModified")]
last_modified: String,
}
#[derive(Serialize)]
#[serde(rename = "Error")]
struct S3Error {
#[serde(rename = "Code")]
code: String,
#[serde(rename = "Message")]
message: String,
}
fn xml_response<T: Serialize>(data: T) -> Response {
let xml = format!(
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n{}",
to_xml(&data).unwrap_or_default()
);
(
StatusCode::OK,
[(axum::http::header::CONTENT_TYPE, "application/xml")],
xml,
)
.into_response()
}
fn error_response(status: StatusCode, code: &str, message: &str) -> Response {
let error = S3Error {
code: code.to_string(),
message: message.to_string(),
};
let xml = format!(
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n{}",
to_xml(&error).unwrap_or_default()
);
(
status,
[(axum::http::header::CONTENT_TYPE, "application/xml")],
xml,
)
.into_response()
}
#[tokio::main]
async fn main() {
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::from_default_env()
.add_directive("nora_storage=info".parse().expect("valid directive")),
)
.init();
let config = Config::load();
fs::create_dir_all(&config.storage.data_dir).expect("Failed to create data directory");
let state = Arc::new(AppState {
config: config.clone(),
});
let app = Router::new()
.route("/", get(list_buckets))
.route("/{bucket}", get(list_objects))
.route("/{bucket}", put(create_bucket))
.route("/{bucket}", delete(delete_bucket))
.route("/{bucket}/{*key}", put(put_object))
.route("/{bucket}/{*key}", get(get_object))
.route("/{bucket}/{*key}", delete(delete_object))
.layer(DefaultBodyLimit::max(config.storage.max_body_size))
.with_state(state);
let addr = format!("{}:{}", config.server.host, config.server.port);
let listener = tokio::net::TcpListener::bind(&addr)
.await
.expect("Failed to bind to address");
info!("nora-storage (S3 compatible) running on http://{}", addr);
axum::serve(listener, app).await.expect("Server error");
}
async fn list_buckets(State(state): State<Arc<AppState>>) -> Response {
let data_dir = &state.config.storage.data_dir;
let entries = match fs::read_dir(data_dir) {
Ok(e) => e,
Err(_) => {
return error_response(
StatusCode::INTERNAL_SERVER_ERROR,
"InternalError",
"Failed to read data",
)
}
};
let bucket_list: Vec<BucketInfo> = entries
.filter_map(|e| e.ok())
.filter(|e| e.path().is_dir())
.filter_map(|e| {
let name = e.file_name().into_string().ok()?;
let modified = e.metadata().ok()?.modified().ok()?;
let datetime: chrono::DateTime<Utc> = modified.into();
Some(BucketInfo {
name,
creation_date: datetime.format("%Y-%m-%dT%H:%M:%SZ").to_string(),
})
})
.collect();
xml_response(ListBucketsResult {
buckets: Buckets {
bucket: bucket_list,
},
})
}
async fn list_objects(State(state): State<Arc<AppState>>, Path(bucket): Path<String>) -> Response {
let bucket_path = format!("{}/{}", state.config.storage.data_dir, bucket);
if !std::path::Path::new(&bucket_path).is_dir() {
return error_response(
StatusCode::NOT_FOUND,
"NoSuchBucket",
"The specified bucket does not exist",
);
}
let objects = collect_files(std::path::Path::new(&bucket_path), "");
xml_response(ListObjectsResult {
name: bucket,
contents: objects,
})
}
fn collect_files(dir: &std::path::Path, prefix: &str) -> Vec<ObjectInfo> {
let mut objects = Vec::new();
if let Ok(entries) = fs::read_dir(dir) {
for entry in entries.filter_map(|e| e.ok()) {
let path = entry.path();
let name = entry.file_name().into_string().unwrap_or_default();
let key = if prefix.is_empty() {
name.clone()
} else {
format!("{}/{}", prefix, name)
};
if path.is_dir() {
objects.extend(collect_files(&path, &key));
} else if let Ok(metadata) = entry.metadata() {
if let Ok(modified) = metadata.modified() {
let datetime: chrono::DateTime<Utc> = modified.into();
objects.push(ObjectInfo {
key,
size: metadata.len(),
last_modified: datetime.format("%Y-%m-%dT%H:%M:%SZ").to_string(),
});
}
}
}
}
objects
}
async fn create_bucket(State(state): State<Arc<AppState>>, Path(bucket): Path<String>) -> Response {
let bucket_path = format!("{}/{}", state.config.storage.data_dir, bucket);
match fs::create_dir(&bucket_path) {
Ok(_) => (StatusCode::OK, "").into_response(),
Err(_) => error_response(
StatusCode::CONFLICT,
"BucketAlreadyExists",
"Bucket already exists",
),
}
}
async fn put_object(
State(state): State<Arc<AppState>>,
Path((bucket, key)): Path<(String, String)>,
body: Bytes,
) -> Response {
let file_path = format!("{}/{}/{}", state.config.storage.data_dir, bucket, key);
if let Some(parent) = std::path::Path::new(&file_path).parent() {
let _ = fs::create_dir_all(parent);
}
match fs::write(&file_path, &body) {
Ok(_) => {
println!("PUT {}/{} ({} bytes)", bucket, key, body.len());
(StatusCode::OK, "").into_response()
}
Err(e) => {
println!("ERROR writing {}/{}: {}", bucket, key, e);
error_response(
StatusCode::INTERNAL_SERVER_ERROR,
"InternalError",
"Failed to write object",
)
}
}
}
async fn get_object(
State(state): State<Arc<AppState>>,
Path((bucket, key)): Path<(String, String)>,
) -> Response {
let file_path = format!("{}/{}/{}", state.config.storage.data_dir, bucket, key);
match fs::read(&file_path) {
Ok(data) => (StatusCode::OK, data).into_response(),
Err(_) => error_response(
StatusCode::NOT_FOUND,
"NoSuchKey",
"The specified key does not exist",
),
}
}
async fn delete_object(
State(state): State<Arc<AppState>>,
Path((bucket, key)): Path<(String, String)>,
) -> Response {
let file_path = format!("{}/{}/{}", state.config.storage.data_dir, bucket, key);
match fs::remove_file(&file_path) {
Ok(_) => {
println!("DELETE {}/{}", bucket, key);
(StatusCode::NO_CONTENT, "").into_response()
}
Err(_) => error_response(
StatusCode::NOT_FOUND,
"NoSuchKey",
"The specified key does not exist",
),
}
}
async fn delete_bucket(State(state): State<Arc<AppState>>, Path(bucket): Path<String>) -> Response {
let bucket_path = format!("{}/{}", state.config.storage.data_dir, bucket);
match fs::remove_dir(&bucket_path) {
Ok(_) => {
println!("DELETE bucket {}", bucket);
(StatusCode::NO_CONTENT, "").into_response()
}
Err(_) => error_response(
StatusCode::CONFLICT,
"BucketNotEmpty",
"The bucket is not empty",
),
}
}

View File

@@ -1,3 +0,0 @@
[toolchain]
channel = "stable"
components = ["clippy", "rustfmt"]

View File

@@ -1,278 +0,0 @@
#!/usr/bin/env bash
# diff-registry.sh — Differential testing: NORA vs reference registry
# Usage:
# ./scripts/diff-registry.sh docker [nora_url]
# ./scripts/diff-registry.sh npm [nora_url]
# ./scripts/diff-registry.sh cargo [nora_url]
# ./scripts/diff-registry.sh pypi [nora_url]
# ./scripts/diff-registry.sh all [nora_url]
#
# Requires: curl, jq, skopeo (for docker), diff
set -euo pipefail
NORA_URL="${2:-http://localhost:5000}"
TMPDIR=$(mktemp -d)
PASS=0
FAIL=0
SKIP=0
cleanup() { rm -rf "$TMPDIR"; }
trap cleanup EXIT
ok() { PASS=$((PASS+1)); echo " PASS: $1"; }
fail() { FAIL=$((FAIL+1)); echo " FAIL: $1"; }
skip() { SKIP=$((SKIP+1)); echo " SKIP: $1"; }
check_tool() {
if ! command -v "$1" &>/dev/null; then
echo "WARNING: $1 not found, some tests will be skipped"
return 1
fi
return 0
}
# --- Docker ---
diff_docker() {
echo "=== Docker Registry V2 ==="
# 1. /v2/ endpoint returns 200 or 401
local status
status=$(curl -s -o /dev/null -w "%{http_code}" "$NORA_URL/v2/")
if [[ "$status" == "200" || "$status" == "401" ]]; then
ok "/v2/ returns $status"
else
fail "/v2/ returns $status (expected 200 or 401)"
fi
# 2. _catalog returns valid JSON with repositories array
local catalog
catalog=$(curl -s "$NORA_URL/v2/_catalog" 2>/dev/null)
if echo "$catalog" | jq -e '.repositories' &>/dev/null; then
ok "/v2/_catalog has .repositories array"
else
fail "/v2/_catalog invalid JSON: $catalog"
fi
# 3. Push+pull roundtrip with skopeo
if check_tool skopeo; then
local test_image="diff-test/alpine"
local test_tag="diff-$(date +%s)"
# Copy a tiny image to NORA (resolves multi-arch to current platform)
if skopeo copy --dest-tls-verify=false \
docker://docker.io/library/alpine:3.20 \
"docker://${NORA_URL#http*://}/$test_image:$test_tag" 2>/dev/null; then
# Verify manifest structure: must have layers[] and config.digest
skopeo inspect --tls-verify=false --raw \
"docker://${NORA_URL#http*://}/$test_image:$test_tag" \
> "$TMPDIR/nora-manifest.json" 2>/dev/null
local has_layers has_config
has_layers=$(jq -e '.layers | length > 0' "$TMPDIR/nora-manifest.json" 2>/dev/null)
has_config=$(jq -e '.config.digest' "$TMPDIR/nora-manifest.json" 2>/dev/null)
if [[ "$has_layers" == "true" && -n "$has_config" ]]; then
ok "Docker push+pull roundtrip: valid manifest with layers"
else
fail "Docker manifest missing layers or config"
jq . "$TMPDIR/nora-manifest.json" 2>/dev/null || true
fi
# Verify blob is retrievable by digest
local first_layer
first_layer=$(jq -r '.layers[0].digest' "$TMPDIR/nora-manifest.json" 2>/dev/null)
if [[ -n "$first_layer" ]]; then
local blob_status
blob_status=$(curl -s -o /dev/null -w "%{http_code}" \
"$NORA_URL/v2/$test_image/blobs/$first_layer")
if [[ "$blob_status" == "200" ]]; then
ok "Docker blob retrievable by digest"
else
fail "Docker blob GET returned $blob_status"
fi
fi
# Check tags/list
local tags
tags=$(curl -s "$NORA_URL/v2/$test_image/tags/list" 2>/dev/null)
if echo "$tags" | jq -e ".tags[] | select(. == \"$test_tag\")" &>/dev/null; then
ok "tags/list contains pushed tag"
else
fail "tags/list missing pushed tag: $tags"
fi
else
fail "skopeo copy to NORA failed"
fi
else
skip "Docker roundtrip (skopeo not installed)"
fi
}
# --- npm ---
diff_npm() {
echo "=== npm Registry ==="
# 1. Package metadata format
local meta
meta=$(curl -s "$NORA_URL/npm/lodash" 2>/dev/null)
local status=$?
if [[ $status -ne 0 ]]; then
skip "npm metadata (no packages published or upstream unavailable)"
return
fi
if echo "$meta" | jq -e '.name' &>/dev/null; then
ok "npm metadata has .name field"
else
skip "npm metadata (no packages or proxy not configured)"
return
fi
# 2. Tarball URLs point to NORA, not upstream
local tarball_url
tarball_url=$(echo "$meta" | jq -r '.versions | to_entries | last | .value.dist.tarball // empty' 2>/dev/null)
if [[ -n "$tarball_url" ]]; then
if echo "$tarball_url" | grep -qvE "registry.npmjs.org|registry.yarnpkg.com"; then
ok "npm tarball URLs rewritten to NORA"
else
fail "npm tarball URL points to upstream: $tarball_url"
fi
else
skip "npm tarball URL check (no versions)"
fi
}
# --- Cargo ---
diff_cargo() {
echo "=== Cargo Sparse Index ==="
# 1. config.json exists and has dl field
local config
config=$(curl -s "$NORA_URL/cargo/index/config.json" 2>/dev/null)
if echo "$config" | jq -e '.dl' &>/dev/null; then
ok "Cargo config.json has .dl field"
else
fail "Cargo config.json missing .dl: $config"
fi
# 2. config.json has api field
if echo "$config" | jq -e '.api' &>/dev/null; then
ok "Cargo config.json has .api field"
else
fail "Cargo config.json missing .api"
fi
}
# --- PyPI ---
diff_pypi() {
echo "=== PyPI Simple API ==="
# 1. /simple/ returns HTML or JSON
local simple_html
simple_html=$(curl -s -H "Accept: text/html" "$NORA_URL/simple/" 2>/dev/null)
if echo "$simple_html" | grep -qi "<!DOCTYPE\|<html\|simple" &>/dev/null; then
ok "/simple/ returns HTML index"
else
skip "/simple/ HTML (no packages published)"
fi
# 2. PEP 691 JSON response
local simple_json
simple_json=$(curl -s -H "Accept: application/vnd.pypi.simple.v1+json" "$NORA_URL/simple/" 2>/dev/null)
if echo "$simple_json" | jq -e '.projects // .meta' &>/dev/null; then
ok "/simple/ PEP 691 JSON works"
else
skip "/simple/ PEP 691 (not supported or empty)"
fi
}
# --- Go ---
diff_go() {
echo "=== Go Module Proxy ==="
# Basic health: try a known module
local status
status=$(curl -s -o /dev/null -w "%{http_code}" "$NORA_URL/go/golang.org/x/text/@v/list" 2>/dev/null)
if [[ "$status" == "200" || "$status" == "404" ]]; then
ok "Go proxy responds ($status)"
else
skip "Go proxy (status: $status)"
fi
}
# --- Raw ---
diff_raw() {
echo "=== Raw Storage ==="
local test_path="diff-test/test-$(date +%s).txt"
local test_content="diff-registry-test"
# 1. PUT + GET roundtrip
local put_status
put_status=$(curl -s -o /dev/null -w "%{http_code}" -X PUT \
-H "Content-Type: text/plain" \
-d "$test_content" \
"$NORA_URL/raw/$test_path" 2>/dev/null)
if [[ "$put_status" == "200" || "$put_status" == "201" ]]; then
local got
got=$(curl -s "$NORA_URL/raw/$test_path" 2>/dev/null)
if [[ "$got" == "$test_content" ]]; then
ok "Raw PUT+GET roundtrip"
else
fail "Raw GET returned different content: '$got'"
fi
# 2. HEAD returns size
local head_status
head_status=$(curl -s -o /dev/null -w "%{http_code}" -I "$NORA_URL/raw/$test_path" 2>/dev/null)
if [[ "$head_status" == "200" ]]; then
ok "Raw HEAD returns 200"
else
fail "Raw HEAD returned $head_status"
fi
# 3. DELETE
curl -s -o /dev/null -X DELETE "$NORA_URL/raw/$test_path" 2>/dev/null
local after_delete
after_delete=$(curl -s -o /dev/null -w "%{http_code}" "$NORA_URL/raw/$test_path" 2>/dev/null)
if [[ "$after_delete" == "404" ]]; then
ok "Raw DELETE works"
else
fail "Raw DELETE: GET after delete returned $after_delete"
fi
elif [[ "$put_status" == "401" ]]; then
skip "Raw PUT (auth required)"
else
fail "Raw PUT returned $put_status"
fi
}
# --- Main ---
case "${1:-all}" in
docker) diff_docker ;;
npm) diff_npm ;;
cargo) diff_cargo ;;
pypi) diff_pypi ;;
go) diff_go ;;
raw) diff_raw ;;
all)
diff_docker
diff_npm
diff_cargo
diff_pypi
diff_go
diff_raw
;;
*)
echo "Usage: $0 {docker|npm|cargo|pypi|go|raw|all} [nora_url]"
exit 1
;;
esac
echo ""
echo "=== Results: $PASS passed, $FAIL failed, $SKIP skipped ==="
[[ $FAIL -eq 0 ]] && exit 0 || exit 1

View File

@@ -1,9 +0,0 @@
[nora]
packages = ["nora-registry"]
engine = "Llvm"
fail-under = 38
out = ["Json", "Html"]
output-dir = "coverage"
test-timeout = "5m"
exclude-files = ["nora-registry/src/ui/*", "nora-registry/src/main.rs", "nora-registry/src/openapi.rs"]
workspace = false

Some files were not shown because too many files have changed in this diff Show More