security: add bounded receive-path secret detection adapter
Assistant: codex Assistant-Model: gpt-6-astra Assistant-Session: 01a12d13-27f2-7163-b8c0-f682af02772c
This commit is contained in:
parent
e395e5e9c1
commit
6d5c9e2abf
6 changed files with 1035 additions and 0 deletions
10
Makefile
10
Makefile
|
|
@ -121,3 +121,13 @@ help: ## Show this help
|
|||
/^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) }' $(MAKEFILE_LIST)
|
||||
|
||||
.PHONY: check-tools check-sops registry-docs evidence-docs runner-docs runner-status forgejo-runner-status forge-observe test-unit check-runner-tools gitea-deploy gitea-ingress-deploy gitea-status help
|
||||
|
||||
##@ Secret detection
|
||||
|
||||
GO ?= go
|
||||
|
||||
test-secret-detection: ## Verify receive-path refusal with an explicitly admitted test scanner
|
||||
@test -n "$${FORGE_TEST_GITLEAKS:-}" || { echo "FORGE_TEST_GITLEAKS must name the admitted test binary" >&2; exit 2; }
|
||||
cd tools/receive-secret-scan && $(GO) test -count=1 ./...
|
||||
|
||||
.PHONY: test-secret-detection
|
||||
|
|
|
|||
107
docs/receive-secret-detection.md
Normal file
107
docs/receive-secret-detection.md
Normal file
|
|
@ -0,0 +1,107 @@
|
|||
# Forge receive-path secret detection
|
||||
|
||||
FORGE-WP-0006 implements the forge-owned part of RMGR-WP-0021-T03 and
|
||||
Custodian CUST-WP-0080 G07. The local Repo Manager scanner remains separate.
|
||||
The source adapter is not evidence of installed server enforcement.
|
||||
|
||||
## Boundary and ownership
|
||||
|
||||
The forge-owned, standalone Linux adapter composes with Forgejo's generated
|
||||
pre-receive dispatcher. It uses pinned Gitleaks 8.30.1 Linux amd64
|
||||
(SHA256 `88f91962aa2f93ac6ab281d553b9e125f5197bbbce38f9f2437f7299c32e5509`).
|
||||
It requires `/usr/bin/git` and `/proc` but no Python runtime, issuer credential,
|
||||
network access, source checkout, or repository-controlled workflow. The current
|
||||
Forgejo 11.0.3 image has no Python interpreter; this keeps the runtime change to
|
||||
two reviewed executables and a hook fragment, subject to native admission.
|
||||
|
||||
The forge owns tool admission, hook installation and recovery. Repo Manager
|
||||
coordinates repository-control coverage; enablement owns reusable CI templates;
|
||||
source repositories own their workflows. Credential classification and
|
||||
revocation remain with existing incident/custody owners.
|
||||
|
||||
Do not overwrite generated `hooks/pre-receive` or `pre-receive.d/gitea`, or enable
|
||||
arbitrary user Git-hook editing. A reviewed additional executable in the existing
|
||||
`pre-receive.d/` dispatcher is the proposed integration. Server writers, frontend
|
||||
merge/edit/import paths, SSH and HTTP pushes, hook resynchronization, pod restart
|
||||
and new repository creation all need separate native acceptance. No admission
|
||||
or production coverage is inferred from disposable bare-repository tests.
|
||||
|
||||
## What is checked
|
||||
|
||||
Input is the exact `old-object new-object ref-name` batch from receive-pack.
|
||||
Malformed, duplicate, oversized, inconsistent or unsupported updates refuse.
|
||||
All updated ref names are scanned without echoing them. Ref exclusions are frozen
|
||||
from the pre-push snapshot; the traversal scans complete introduced reachable
|
||||
objects across every new tip and every parent, including annotated tags and
|
||||
direct object tags. A secret added and removed before the tip is still examined.
|
||||
|
||||
Blob data, raw commit/tag contents and tree names are scanned. Symlink contents
|
||||
are read as Git blobs; their filesystem targets are never followed. Changed trees
|
||||
containing external gitlinks refuse because the referenced repository is not
|
||||
covered. Git replacement objects and grafts cannot hide history. Partial/shallow
|
||||
repositories, missing objects and unverified traversal refuse. Receive-pack's
|
||||
quarantine object locations are preserved only for Git reads; the adapter never
|
||||
updates refs or copies objects into the primary store.
|
||||
|
||||
The scanner executable is opened without following a final symlink, checked for
|
||||
ownership/mode/regular-file shape and exact SHA256, and executed through its open
|
||||
file descriptor. Scanner config and ignore files are private temporary files;
|
||||
repository configuration, inline allow comments and ambient scanner settings do
|
||||
not disable detection. Child diagnostics/findings are discarded. Output contains
|
||||
only a fixed status/reason, counts, version and, for an object detection, its ID.
|
||||
Privately review the identified object; never paste its contents into an issue,
|
||||
State Hub, CI logs or chat. Standard Git output may independently display ref
|
||||
names; the adapter does not make Git's transport diagnostics a secret-safe log.
|
||||
|
||||
Exit 0 is `clean` in this declared scope; exit 1 is `blocked`; exit 2 is
|
||||
`unverified`. Both nonzero statuses must refuse the complete push. There is no
|
||||
implicit baseline/ignore or override switch. A false-positive exception needs a
|
||||
separate reviewed mechanism with exact scope, approval and expiry.
|
||||
|
||||
## Bounds and remaining coverage
|
||||
|
||||
Current bounds: 256 updates, 10,000 existing refs, 2,000 introduced objects,
|
||||
8 MiB per object, 64 MiB total, 512 KiB update input and 90 seconds for the check.
|
||||
Scanner calls also use the scanner's 15-second timeout. Object sizes and total
|
||||
bytes are checked before scanning payloads. Limits reject rather
|
||||
than skip. Large imports need an admitted path; do not disable the gate to pass
|
||||
an unreviewed import. Git/scanner versions and limits require native load testing.
|
||||
|
||||
The adapter does not rescan objects already reachable before the push, scan
|
||||
unreachable extra pack objects, LFS payloads, other repositories, runtime logs,
|
||||
artifacts or credentials outside Git. It is not an authorization boundary against
|
||||
a forge administrator or hostile same-UID process. Default scanner rules remain
|
||||
heuristic. Historical discovery and finding-to-response are independent work;
|
||||
removing a source value or rejecting a push does not revoke an exposed credential.
|
||||
CUST-WP-0073-T05's founder-deferred rotation is unchanged.
|
||||
|
||||
The 165 observed on-disk repository directories are a discovery count, not an
|
||||
admitted production-software denominator. The 22 reviewed repositories provide a
|
||||
named rollout starting scope; the owner must reconcile all production repositories,
|
||||
wikis, archives, mirrors and new-repository lifecycle before claiming DSM coverage.
|
||||
|
||||
## Build and disposable acceptance
|
||||
|
||||
Use the admitted Go toolchain (the current candidate uses official Go 1.26.9,
|
||||
verified against the SHA256 from go.dev release metadata). No third-party Go
|
||||
modules are required. Keep binaries and logs outside the repository:
|
||||
|
||||
```sh
|
||||
cd tools/receive-secret-scan
|
||||
FORGE_TEST_GITLEAKS=/reviewed/gitleaks go test -count=1 -v ./...
|
||||
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -buildvcs=false -o /tmp/receive-secret-scan .
|
||||
```
|
||||
|
||||
The root `make test-secret-detection` target requires `FORGE_TEST_GITLEAKS`
|
||||
and runs this module; set `GO` to the admitted compiler path where needed.
|
||||
|
||||
Tests use disposable Git servers, actual receive-pack quarantine and generated
|
||||
keys. Scanner-dependent tests explicitly skip without `FORGE_TEST_GITLEAKS`;
|
||||
skips are not acceptance. Native activation additionally needs exact original
|
||||
hook hashes, tool provenance, dry-run/diff, abort owner, rollback and value-free
|
||||
receipts under the accepted build-mode change rule.
|
||||
|
||||
References: [Git receive-pack and quarantine](https://git-scm.com/docs/git-receive-pack),
|
||||
[Git object traversal](https://git-scm.com/docs/git-rev-list),
|
||||
[Forgejo 11.0.3 generated hooks](https://codeberg.org/forgejo/forgejo/src/tag/v11.0.3/modules/repository/hooks.go),
|
||||
[Gitleaks release](https://github.com/gitleaks/gitleaks/releases/tag/v8.30.1).
|
||||
3
tools/receive-secret-scan/go.mod
Normal file
3
tools/receive-secret-scan/go.mod
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
module railiance-forge/receive-secret-scan
|
||||
|
||||
go 1.22
|
||||
404
tools/receive-secret-scan/main.go
Normal file
404
tools/receive-secret-scan/main.go
Normal file
|
|
@ -0,0 +1,404 @@
|
|||
// receive-secret-scan is a content-silent, Linux pre-receive control adapter.
|
||||
// It never changes refs, runs repository programs, or retrieves credentials.
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
const scannerHash = "88f91962aa2f93ac6ab281d553b9e125f5197bbbce38f9f2437f7299c32e5509"
|
||||
const scannerVersion = "8.30.1"
|
||||
const maxInput = 512 * 1024
|
||||
const maxObject = 8 * 1024 * 1024
|
||||
const maxTotal = 64 * 1024 * 1024
|
||||
const maxObjects = 2000
|
||||
const maxRefs = 10000
|
||||
|
||||
var oidPattern = regexp.MustCompile(`^(?:[0-9a-f]{40}|[0-9a-f]{64})$`)
|
||||
var errLimit = errors.New("output-limit")
|
||||
|
||||
type result struct {
|
||||
Status string `json:"status"`
|
||||
Scope string `json:"scope"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
Scanned int `json:"scanned_objects"`
|
||||
ScannedRefs int `json:"scanned_refs"`
|
||||
BlockedID string `json:"blocked_object_id,omitempty"`
|
||||
Version string `json:"scanner_version"`
|
||||
}
|
||||
type update struct{ Old, New, Ref string }
|
||||
type cappedBuffer struct {
|
||||
bytes.Buffer
|
||||
limit int
|
||||
}
|
||||
|
||||
func (b *cappedBuffer) Write(p []byte) (int, error) {
|
||||
if len(p) > b.limit-b.Len() {
|
||||
return 0, errLimit
|
||||
}
|
||||
return b.Buffer.Write(p)
|
||||
}
|
||||
func zero(s string) bool { return strings.Trim(s, "0") == "" }
|
||||
func parseUpdates(raw []byte) ([]update, error) {
|
||||
if len(raw) == 0 || len(raw) > maxInput || raw[len(raw)-1] != '\n' {
|
||||
return nil, errors.New("invalid-update-input")
|
||||
}
|
||||
lines := strings.Split(string(raw[:len(raw)-1]), "\n")
|
||||
if len(lines) > 256 {
|
||||
return nil, errors.New("update-limit")
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
var out []update
|
||||
for _, line := range lines {
|
||||
fields := strings.Split(line, " ")
|
||||
if len(fields) != 3 || !oidPattern.MatchString(fields[0]) || !oidPattern.MatchString(fields[1]) || len(fields[0]) != len(fields[1]) || len(fields[2]) > 1024 || !strings.HasPrefix(fields[2], "refs/") || strings.ContainsAny(fields[2], "\x00\r\n\t") || seen[fields[2]] || zero(fields[0]) && zero(fields[1]) {
|
||||
return nil, errors.New("invalid-update-input")
|
||||
}
|
||||
seen[fields[2]] = true
|
||||
out = append(out, update{fields[0], fields[1], fields[2]})
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Only receive-pack object-location variables cross into Git. Scanner processes
|
||||
// get none. Never inherit arbitrary GIT_CONFIG_* or credential/provider settings.
|
||||
func gitEnvironment() []string {
|
||||
env := []string{"PATH=/usr/local/bin:/usr/bin:/bin", "HOME=/nonexistent", "LANG=C", "GIT_CONFIG_NOSYSTEM=1", "GIT_CONFIG_GLOBAL=/dev/null", "GIT_NO_REPLACE_OBJECTS=1", "GIT_GRAFT_FILE=/dev/null", "GIT_NO_LAZY_FETCH=1", "GIT_TERMINAL_PROMPT=0", "GIT_ASKPASS=/bin/false", "GIT_SSH_COMMAND=/bin/false"}
|
||||
for _, key := range []string{"GIT_DIR", "GIT_COMMON_DIR", "GIT_OBJECT_DIRECTORY", "GIT_ALTERNATE_OBJECT_DIRECTORIES", "GIT_QUARANTINE_PATH"} {
|
||||
if val, ok := os.LookupEnv(key); ok {
|
||||
env = append(env, key+"="+val)
|
||||
}
|
||||
}
|
||||
return env
|
||||
}
|
||||
func git(ctx context.Context, input []byte, limit int, args ...string) ([]byte, error) {
|
||||
cmd := exec.CommandContext(ctx, "/usr/bin/git", args...)
|
||||
cmd.Env = gitEnvironment()
|
||||
cmd.Stdin = bytes.NewReader(input)
|
||||
b := &cappedBuffer{limit: limit}
|
||||
cmd.Stdout = b
|
||||
cmd.Stderr = io.Discard
|
||||
if cmd.Run() != nil {
|
||||
return nil, errors.New("git-" + args[0] + "-failed")
|
||||
}
|
||||
return b.Bytes(), nil
|
||||
}
|
||||
func refs(ctx context.Context) ([]byte, map[string]string, error) {
|
||||
raw, err := git(ctx, nil, 2*1024*1024, "for-each-ref", "--format=%(objectname) %(refname)")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
found := map[string]string{}
|
||||
for _, line := range strings.Split(strings.TrimSuffix(string(raw), "\n"), "\n") {
|
||||
if line == "" {
|
||||
continue
|
||||
}
|
||||
parts := strings.Split(line, " ")
|
||||
if len(parts) != 2 || !oidPattern.MatchString(parts[0]) || !strings.HasPrefix(parts[1], "refs/") {
|
||||
return nil, nil, errors.New("invalid-ref-snapshot")
|
||||
}
|
||||
found[parts[1]] = parts[0]
|
||||
}
|
||||
if len(found) > maxRefs {
|
||||
return nil, nil, errors.New("ref-limit")
|
||||
}
|
||||
return raw, found, nil
|
||||
}
|
||||
func pinScanner(path string) (*os.File, error) {
|
||||
fd, err := syscall.Open(path, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK, 0)
|
||||
if err != nil {
|
||||
return nil, errors.New("scanner-file-refused")
|
||||
}
|
||||
f := os.NewFile(uintptr(fd), path)
|
||||
ok := false
|
||||
defer func() {
|
||||
if !ok {
|
||||
f.Close()
|
||||
}
|
||||
}()
|
||||
st, err := f.Stat()
|
||||
if err != nil {
|
||||
return nil, errors.New("scanner-file-refused")
|
||||
}
|
||||
native, valid := st.Sys().(*syscall.Stat_t)
|
||||
if !valid || !st.Mode().IsRegular() || st.Mode().Perm()&022 != 0 || st.Mode().Perm()&0111 == 0 || st.Size() > 64*1024*1024 || native.Uid != 0 && native.Uid != uint32(os.Getuid()) {
|
||||
return nil, errors.New("scanner-file-refused")
|
||||
}
|
||||
hash := sha256.New()
|
||||
if _, err = io.Copy(hash, f); err != nil || hex.EncodeToString(hash.Sum(nil)) != scannerHash {
|
||||
return nil, errors.New("scanner-digest-mismatch")
|
||||
}
|
||||
ok = true
|
||||
return f, nil
|
||||
}
|
||||
func scan(ctx context.Context, f *os.File, dir string, content []byte) (bool, error) {
|
||||
cmd := exec.CommandContext(ctx, "/proc/self/fd/3", "stdin", "--config", filepath.Join(dir, "scanner.toml"), "--gitleaks-ignore-path", filepath.Join(dir, "ignore"), "--ignore-gitleaks-allow", "--redact=100", "--no-banner", "--no-color", "--log-level", "fatal", "--exit-code", "20", "--timeout", "15")
|
||||
cmd.ExtraFiles = []*os.File{f}
|
||||
cmd.Dir = dir
|
||||
cmd.Env = []string{"PATH=/usr/bin:/bin", "HOME=" + dir, "TMPDIR=" + dir, "LANG=C"}
|
||||
cmd.Stdin = bytes.NewReader(content)
|
||||
cmd.Stdout = io.Discard
|
||||
cmd.Stderr = io.Discard
|
||||
err := cmd.Run()
|
||||
if err == nil {
|
||||
return false, nil
|
||||
}
|
||||
var exit *exec.ExitError
|
||||
if errors.As(err, &exit) && exit.ExitCode() == 20 {
|
||||
return true, nil
|
||||
}
|
||||
return false, errors.New("scanner-failed")
|
||||
}
|
||||
|
||||
// Tree object names are covered too. Gitlinks refer to an external repository,
|
||||
// so this version refuses them rather than claiming transitive coverage.
|
||||
func treeContent(raw []byte, oidBytes int) ([]byte, error) {
|
||||
var names []byte
|
||||
for len(raw) > 0 {
|
||||
space := bytes.IndexByte(raw, ' ')
|
||||
nul := bytes.IndexByte(raw, 0)
|
||||
if space <= 0 || nul <= space || len(raw) < nul+1+oidBytes {
|
||||
return nil, errors.New("invalid-tree")
|
||||
}
|
||||
mode := string(raw[:space])
|
||||
if mode == "160000" {
|
||||
return nil, errors.New("external-gitlink-unverified")
|
||||
}
|
||||
if mode != "100644" && mode != "100755" && mode != "120000" && mode != "40000" {
|
||||
return nil, errors.New("unsupported-tree-entry")
|
||||
}
|
||||
names = append(names, raw[space+1:nul]...)
|
||||
names = append(names, '\n')
|
||||
raw = raw[nul+1+oidBytes:]
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
func check(ctx context.Context, updates []update, binary string) (out result) {
|
||||
out = result{Status: "unverified", Scope: "introduced-reachable-git-objects-and-updated-ref-names", Version: scannerVersion}
|
||||
fail := func(err error) result { out.Reason = err.Error(); return out }
|
||||
scanner, err := pinScanner(binary)
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
defer scanner.Close()
|
||||
format, err := git(ctx, nil, 128, "rev-parse", "--show-object-format")
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
width := 40
|
||||
if string(format) == "sha256\n" {
|
||||
width = 64
|
||||
} else if string(format) != "sha1\n" {
|
||||
return fail(errors.New("unsupported-object-format"))
|
||||
}
|
||||
shallow, err := git(ctx, nil, 128, "rev-parse", "--is-shallow-repository")
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
if string(shallow) != "false\n" {
|
||||
return fail(errors.New("shallow-repository-unverified"))
|
||||
}
|
||||
// Partial repositories may perform network retrieval while reading objects.
|
||||
config, err := git(ctx, nil, 1024*1024, "config", "--local", "--includes", "--name-only", "--list")
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
for _, key := range strings.Split(strings.ToLower(string(config)), "\n") {
|
||||
if key == "extensions.partialclone" || strings.HasSuffix(key, ".promisor") {
|
||||
return fail(errors.New("partial-repository-unverified"))
|
||||
}
|
||||
}
|
||||
snapshot, existing, err := refs(ctx)
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
tips := map[string]bool{}
|
||||
for _, u := range updates {
|
||||
if len(u.Old) != width {
|
||||
return fail(errors.New("object-format-mismatch"))
|
||||
}
|
||||
if _, err = git(ctx, nil, 128, "check-ref-format", u.Ref); err != nil {
|
||||
return fail(errors.New("invalid-ref-name"))
|
||||
}
|
||||
current, exists := existing[u.Ref]
|
||||
if zero(u.Old) && exists || !zero(u.Old) && current != u.Old {
|
||||
return fail(errors.New("old-ref-mismatch"))
|
||||
}
|
||||
if !zero(u.New) {
|
||||
tips[u.New] = true
|
||||
}
|
||||
}
|
||||
// Freeze exclusions from the pre-push snapshot, never a mutable --all read.
|
||||
// Scan all introduced reachable objects, not just a tip diff or first parent.
|
||||
var revisions strings.Builder
|
||||
for tip := range tips {
|
||||
revisions.WriteString(tip + "\n")
|
||||
}
|
||||
objects := map[string]bool{}
|
||||
if len(tips) > 0 {
|
||||
excluded := map[string]bool{}
|
||||
for _, oid := range existing {
|
||||
excluded[oid] = true
|
||||
}
|
||||
for oid := range excluded {
|
||||
revisions.WriteString("^" + oid + "\n")
|
||||
}
|
||||
raw, e := git(ctx, []byte(revisions.String()), (maxObjects+1)*(width+1), "rev-list", "--objects", "--no-object-names", "--stdin")
|
||||
if e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
for _, oid := range strings.Fields(string(raw)) {
|
||||
if !oidPattern.MatchString(oid) || len(oid) != width {
|
||||
return fail(errors.New("invalid-object-list"))
|
||||
}
|
||||
objects[oid] = true
|
||||
}
|
||||
}
|
||||
if len(objects) > maxObjects {
|
||||
return fail(errors.New("object-limit"))
|
||||
}
|
||||
ids := make([]string, 0, len(objects))
|
||||
for oid := range objects {
|
||||
ids = append(ids, oid)
|
||||
}
|
||||
sort.Strings(ids)
|
||||
var metadata []byte
|
||||
if len(ids) > 0 {
|
||||
metadata, err = git(ctx, []byte(strings.Join(ids, "\n")+"\n"), (maxObjects+1)*100, "cat-file", "--batch-check=%(objectname) %(objecttype) %(objectsize)")
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
}
|
||||
lines := strings.Split(strings.TrimSuffix(string(metadata), "\n"), "\n")
|
||||
if len(ids) == 0 {
|
||||
lines = nil
|
||||
}
|
||||
if len(lines) != len(ids) {
|
||||
return fail(errors.New("incomplete-object-metadata"))
|
||||
}
|
||||
// Reject oversized batches before reading/scanning any object payload.
|
||||
type objectInfo struct {
|
||||
kind string
|
||||
size int64
|
||||
}
|
||||
var objectMetadata []objectInfo
|
||||
total := int64(0)
|
||||
for i, line := range lines {
|
||||
parts := strings.Split(line, " ")
|
||||
if len(parts) != 3 || parts[0] != ids[i] {
|
||||
return fail(errors.New("invalid-object-metadata"))
|
||||
}
|
||||
kind := parts[1]
|
||||
if kind != "blob" && kind != "commit" && kind != "tag" && kind != "tree" {
|
||||
return fail(errors.New("unsupported-object-type"))
|
||||
}
|
||||
size, e := strconv.ParseInt(parts[2], 10, 64)
|
||||
if e != nil || size < 0 || size > maxObject || total+size > maxTotal {
|
||||
return fail(errors.New("object-byte-limit"))
|
||||
}
|
||||
total += size
|
||||
objectMetadata = append(objectMetadata, objectInfo{kind, size})
|
||||
}
|
||||
dir, err := os.MkdirTemp("", "forge-secret-scan-")
|
||||
if err != nil {
|
||||
return fail(errors.New("temporary-directory-unavailable"))
|
||||
}
|
||||
defer os.RemoveAll(dir)
|
||||
if os.WriteFile(filepath.Join(dir, "scanner.toml"), []byte("[extend]\nuseDefault = true\n"), 0600) != nil || os.WriteFile(filepath.Join(dir, "ignore"), nil, 0600) != nil {
|
||||
return fail(errors.New("scanner-config-unavailable"))
|
||||
}
|
||||
// Ref names can themselves contain recognized tokens. Never echo them.
|
||||
for _, u := range updates {
|
||||
blocked, e := scan(ctx, scanner, dir, []byte(u.Ref))
|
||||
if e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
out.ScannedRefs++
|
||||
if blocked {
|
||||
out.Status = "blocked"
|
||||
out.Reason = "ref-name-detection"
|
||||
return out
|
||||
}
|
||||
}
|
||||
for i, info := range objectMetadata {
|
||||
kind, size := info.kind, info.size
|
||||
raw, e := git(ctx, nil, int(size)+1, "cat-file", kind, ids[i])
|
||||
if e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
if int64(len(raw)) != size {
|
||||
return fail(errors.New("object-size-mismatch"))
|
||||
}
|
||||
if kind == "tree" {
|
||||
raw, e = treeContent(raw, width/2)
|
||||
if e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
}
|
||||
blocked, e := scan(ctx, scanner, dir, raw)
|
||||
if e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
out.Scanned++
|
||||
if blocked {
|
||||
out.Status = "blocked"
|
||||
out.BlockedID = ids[i]
|
||||
return out
|
||||
}
|
||||
}
|
||||
after, _, err := refs(ctx)
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
if !bytes.Equal(snapshot, after) {
|
||||
return fail(errors.New("refs-changed-during-scan"))
|
||||
}
|
||||
out.Status = "clean"
|
||||
return out
|
||||
}
|
||||
func main() {
|
||||
flags := flag.NewFlagSet("receive-secret-scan", flag.ContinueOnError)
|
||||
flags.SetOutput(io.Discard)
|
||||
binary := flags.String("scanner", "", "reviewed Gitleaks executable")
|
||||
out := result{Status: "unverified", Scope: "introduced-reachable-git-objects-and-updated-ref-names", Version: scannerVersion, Reason: "invalid-invocation"}
|
||||
if flags.Parse(os.Args[1:]) == nil && *binary != "" && flags.NArg() == 0 {
|
||||
raw, err := io.ReadAll(io.LimitReader(os.Stdin, maxInput+1))
|
||||
if err == nil {
|
||||
updates, e := parseUpdates(raw)
|
||||
if e != nil {
|
||||
out.Reason = e.Error()
|
||||
} else {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
out = check(ctx, updates, *binary)
|
||||
cancel()
|
||||
}
|
||||
}
|
||||
}
|
||||
// Only fixed reason codes, counts and an object ID may reach the caller.
|
||||
encoded, _ := json.Marshal(out)
|
||||
fmt.Println(string(encoded))
|
||||
if out.Status == "clean" {
|
||||
return
|
||||
}
|
||||
if out.Status == "blocked" {
|
||||
os.Exit(1)
|
||||
}
|
||||
os.Exit(2)
|
||||
}
|
||||
425
tools/receive-secret-scan/main_test.go
Normal file
425
tools/receive-secret-scan/main_test.go
Normal file
|
|
@ -0,0 +1,425 @@
|
|||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"crypto/x509"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"encoding/pem"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestUpdateInputRefusesAmbiguity(t *testing.T) {
|
||||
z := strings.Repeat("0", 40)
|
||||
a := strings.Repeat("a", 40)
|
||||
valid := z + " " + a + " refs/heads/main\n"
|
||||
for _, s := range []string{"", "\n", strings.TrimSuffix(valid, "\n"), z + " " + z + " refs/heads/x\n", valid + valid, z + " " + a + " --all\n", z + " " + a + " refs/heads/a b\n", strings.Repeat(valid, 257)} {
|
||||
if _, err := parseUpdates([]byte(s)); err == nil {
|
||||
t.Fatal("accepted invalid input")
|
||||
}
|
||||
}
|
||||
if _, err := parseUpdates([]byte(valid)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestTreeRefusesMalformedAndExternalEntries(t *testing.T) {
|
||||
for _, raw := range [][]byte{[]byte("bad"), []byte("100644 a\x00short"), append([]byte("160000 submodule\x00"), make([]byte, 20)...)} {
|
||||
if _, err := treeContent(raw, 20); err == nil {
|
||||
t.Fatal("accepted unsupported tree")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestGitEnvironmentDoesNotInheritConfigOrSecrets(t *testing.T) {
|
||||
t.Setenv("GIT_CONFIG_COUNT", "1")
|
||||
t.Setenv("GIT_CONFIG_KEY_0", "core.hooksPath")
|
||||
t.Setenv("GIT_CONFIG_VALUE_0", "/untrusted")
|
||||
t.Setenv("SCANNER_CREDENTIAL", "synthetic")
|
||||
t.Setenv("GIT_OBJECT_DIRECTORY", "/reviewed/quarantine")
|
||||
env := strings.Join(gitEnvironment(), "\n")
|
||||
if strings.Contains(env, "untrusted") || strings.Contains(env, "CREDENTIAL") || strings.Contains(env, "GIT_CONFIG_COUNT") || !strings.Contains(env, "GIT_OBJECT_DIRECTORY=/reviewed/quarantine") {
|
||||
t.Fatal("environment boundary failed")
|
||||
}
|
||||
}
|
||||
func TestCappedOutput(t *testing.T) {
|
||||
b := &cappedBuffer{limit: 3}
|
||||
if _, err := b.Write([]byte("four")); err == nil || b.Len() != 0 {
|
||||
t.Fatal("unbounded output")
|
||||
}
|
||||
}
|
||||
func run(t *testing.T, dir string, input []byte, args ...string) []byte {
|
||||
t.Helper()
|
||||
cmd := exec.Command(args[0], args[1:]...)
|
||||
cmd.Dir = dir
|
||||
cmd.Stdin = bytes.NewReader(input)
|
||||
cmd.Env = append(os.Environ(), "GIT_CONFIG_NOSYSTEM=1", "GIT_CONFIG_GLOBAL=/dev/null", "GIT_TERMINAL_PROMPT=0")
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("disposable command failed (output withheld), executable %s", filepath.Base(args[0]))
|
||||
}
|
||||
return out
|
||||
}
|
||||
func write(t *testing.T, path string, b []byte) {
|
||||
t.Helper()
|
||||
if os.WriteFile(path, b, 0600) != nil {
|
||||
t.Fatal("fixture write failed")
|
||||
}
|
||||
}
|
||||
func key(t *testing.T) []byte {
|
||||
_, priv, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
der, err := x509.MarshalPKCS8PrivateKey(priv)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: der})
|
||||
}
|
||||
func binary(t *testing.T) (string, string) {
|
||||
t.Helper()
|
||||
scanner := os.Getenv("FORGE_TEST_GITLEAKS")
|
||||
if scanner == "" {
|
||||
t.Skip("requires explicitly admitted disposable Gitleaks binary")
|
||||
}
|
||||
if _, err := os.Stat(scanner); err != nil {
|
||||
t.Fatal("configured test scanner missing")
|
||||
}
|
||||
path := filepath.Join(t.TempDir(), "adapter")
|
||||
run(t, "", nil, "go", "build", "-buildvcs=false", "-o", path, ".")
|
||||
return path, scanner
|
||||
}
|
||||
|
||||
type fixture struct{ client, server, adapter, scanner string }
|
||||
|
||||
func setup(t *testing.T, adapter, scanner string) fixture {
|
||||
t.Helper()
|
||||
area := t.TempDir()
|
||||
f := fixture{filepath.Join(area, "client"), filepath.Join(area, "remote.git"), adapter, scanner}
|
||||
run(t, area, nil, "git", "init", "-q", "--bare", f.server)
|
||||
run(t, area, nil, "git", "init", "-q", "-b", "main", f.client)
|
||||
for _, dir := range []string{f.client, f.server} {
|
||||
run(t, dir, nil, "git", "config", "core.hooksPath", filepath.Join(dir, "empty-hooks"))
|
||||
}
|
||||
run(t, f.client, nil, "git", "config", "user.name", "Disposable Test")
|
||||
run(t, f.client, nil, "git", "config", "user.email", "probe@example.test")
|
||||
f.commit(t, "source.txt", []byte("clean baseline\n"), "baseline")
|
||||
run(t, f.client, nil, "git", "push", "-q", f.server, "HEAD:refs/heads/main")
|
||||
run(t, f.server, nil, "git", "config", "core.hooksPath", filepath.Join(f.server, "hooks"))
|
||||
f.install(t, scanner)
|
||||
return f
|
||||
}
|
||||
func (f fixture) commit(t *testing.T, name string, content []byte, message string) string {
|
||||
t.Helper()
|
||||
write(t, filepath.Join(f.client, name), content)
|
||||
run(t, f.client, nil, "git", "add", "--", name)
|
||||
run(t, f.client, []byte(message), "git", "commit", "-q", "-F", "-")
|
||||
return strings.TrimSpace(string(run(t, f.client, nil, "git", "rev-parse", "HEAD")))
|
||||
}
|
||||
func quote(s string) string { return "'" + strings.ReplaceAll(s, "'", "'\\''") + "'" }
|
||||
func (f fixture) install(t *testing.T, scanner string) {
|
||||
t.Helper()
|
||||
p := filepath.Join(f.server, "hooks", "pre-receive")
|
||||
write(t, p, []byte("#!/bin/sh\nexec "+quote(f.adapter)+" --scanner "+quote(scanner)+"\n"))
|
||||
if os.Chmod(p, 0755) != nil {
|
||||
t.Fatal("chmod")
|
||||
}
|
||||
}
|
||||
func (f fixture) push(t *testing.T, want int, args ...string) []byte {
|
||||
t.Helper()
|
||||
full := append([]string{"push", "--porcelain", f.server}, args...)
|
||||
cmd := exec.Command("git", full...)
|
||||
cmd.Dir = f.client
|
||||
cmd.Env = append(os.Environ(), "GIT_CONFIG_NOSYSTEM=1", "GIT_CONFIG_GLOBAL=/dev/null")
|
||||
out, err := cmd.CombinedOutput()
|
||||
var observed result
|
||||
for _, line := range bytes.Split(out, []byte("\n")) {
|
||||
data := regexp.MustCompile(`\{"status":.*\}`).Find(line)
|
||||
if len(data) > 0 {
|
||||
_ = json.Unmarshal(data, &observed)
|
||||
}
|
||||
}
|
||||
if want == 0 && err != nil || want != 0 && err == nil {
|
||||
t.Fatalf("unexpected push outcome, want %d; status=%s reason=%s", want, observed.Status, observed.Reason)
|
||||
}
|
||||
expected := "clean"
|
||||
if want == 1 {
|
||||
expected = "blocked"
|
||||
}
|
||||
if want == 2 {
|
||||
expected = "unverified"
|
||||
}
|
||||
if observed.Status != expected {
|
||||
t.Fatalf("wrong control result, want %s; status=%s reason=%s", expected, observed.Status, observed.Reason)
|
||||
}
|
||||
if bytes.Contains(out, []byte("PRIVATE KEY")) {
|
||||
t.Fatal("synthetic content leaked")
|
||||
}
|
||||
return out
|
||||
}
|
||||
func TestNativePushBoundaries(t *testing.T) {
|
||||
adapter, scanner := binary(t)
|
||||
t.Run("clean-existing-and-new-branch-and-deletion", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
f.commit(t, "source.txt", []byte("changed clean\n"), "clean")
|
||||
f.push(t, 0, "HEAD:refs/heads/main")
|
||||
f.push(t, 0, "HEAD:refs/heads/feature")
|
||||
f.push(t, 0, ":refs/heads/feature")
|
||||
})
|
||||
t.Run("initial-repository-clean-and-secret", func(t *testing.T) {
|
||||
for _, secret := range []bool{false, true} {
|
||||
t.Run(fmt.Sprint(secret), func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
fresh := filepath.Join(t.TempDir(), "fresh.git")
|
||||
run(t, "", nil, "git", "init", "-q", "--bare", fresh)
|
||||
run(t, fresh, nil, "git", "config", "core.hooksPath", filepath.Join(fresh, "hooks"))
|
||||
f.server = fresh
|
||||
f.install(t, scanner)
|
||||
want := 0
|
||||
if secret {
|
||||
f.commit(t, "secret.txt", key(t), "fixture")
|
||||
want = 1
|
||||
}
|
||||
f.push(t, want, "HEAD:refs/heads/main")
|
||||
})
|
||||
}
|
||||
})
|
||||
t.Run("sha256-object-format", func(t *testing.T) {
|
||||
t.Setenv("GIT_DEFAULT_HASH", "sha256")
|
||||
f := setup(t, adapter, scanner)
|
||||
f.commit(t, "source.txt", []byte("clean change\n"), "clean")
|
||||
f.push(t, 0, "HEAD:refs/heads/main")
|
||||
f.commit(t, "secret.txt", key(t), "fixture")
|
||||
f.push(t, 1, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("secret-in-tip", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
oid := f.commit(t, "secret.txt", key(t), "synthetic fixture")
|
||||
output := f.push(t, 1, "HEAD:refs/heads/main")
|
||||
if !bytes.Contains(output, []byte(`"status":"blocked"`)) {
|
||||
t.Fatal("not a detection")
|
||||
}
|
||||
c := exec.Command("git", "cat-file", "-e", oid)
|
||||
c.Dir = f.server
|
||||
if c.Run() == nil {
|
||||
t.Fatal("rejected quarantined commit entered store")
|
||||
}
|
||||
})
|
||||
t.Run("secret-added-then-deleted", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
f.commit(t, "secret.txt", key(t), "temporary fixture")
|
||||
run(t, f.client, nil, "git", "rm", "-q", "secret.txt")
|
||||
run(t, f.client, nil, "git", "commit", "-qm", "removed fixture")
|
||||
out := f.push(t, 1, "HEAD:refs/heads/main")
|
||||
if !bytes.Contains(out, []byte(`"status":"blocked"`)) {
|
||||
t.Fatal("history not scanned")
|
||||
}
|
||||
})
|
||||
t.Run("new-branch-and-force-update", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
f.commit(t, "secret.txt", key(t), "fixture")
|
||||
f.push(t, 1, "HEAD:refs/heads/new")
|
||||
f.push(t, 1, "+HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("merge-second-parent-history", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
run(t, f.client, nil, "git", "checkout", "-qb", "side")
|
||||
f.commit(t, "secret.txt", key(t), "fixture")
|
||||
run(t, f.client, nil, "git", "rm", "-q", "secret.txt")
|
||||
run(t, f.client, nil, "git", "commit", "-qm", "remove fixture")
|
||||
run(t, f.client, nil, "git", "checkout", "-q", "main")
|
||||
f.commit(t, "main.txt", []byte("clean\n"), "main")
|
||||
run(t, f.client, nil, "git", "merge", "-q", "--no-ff", "-m", "merge", "side")
|
||||
f.push(t, 1, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("commit-message", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
f.commit(t, "source.txt", []byte("clean\n"), string(key(t)))
|
||||
f.push(t, 1, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("annotated-tag-message", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
run(t, f.client, key(t), "git", "tag", "-a", "synthetic", "-F", "-")
|
||||
f.push(t, 1, "refs/tags/synthetic")
|
||||
})
|
||||
t.Run("tag-direct-to-blob", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
oid := strings.TrimSpace(string(run(t, f.client, key(t), "git", "hash-object", "-w", "--stdin")))
|
||||
run(t, f.client, nil, "git", "update-ref", "refs/tags/blob", oid)
|
||||
f.push(t, 1, "refs/tags/blob")
|
||||
})
|
||||
t.Run("multi-ref-all-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
run(t, f.client, nil, "git", "branch", "clean")
|
||||
f.commit(t, "secret.txt", key(t), "fixture")
|
||||
f.push(t, 1, "clean:refs/heads/clean-new", "HEAD:refs/heads/main")
|
||||
c := exec.Command("git", "show-ref", "--verify", "refs/heads/clean-new")
|
||||
c.Dir = f.server
|
||||
if c.Run() == nil {
|
||||
t.Fatal("partial ref update")
|
||||
}
|
||||
})
|
||||
t.Run("missing-scanner-and-deletion-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
f.install(t, filepath.Join(t.TempDir(), "missing"))
|
||||
f.commit(t, "source.txt", []byte("clean\n"), "clean")
|
||||
out := f.push(t, 2, "HEAD:refs/heads/main")
|
||||
if !bytes.Contains(out, []byte(`"status":"unverified"`)) {
|
||||
t.Fatal("missing scanner accepted")
|
||||
}
|
||||
f.push(t, 2, ":refs/heads/main")
|
||||
})
|
||||
t.Run("wrong-executable-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
fake := filepath.Join(t.TempDir(), "fake")
|
||||
write(t, fake, []byte("#!/bin/sh\nexit 0\n"))
|
||||
os.Chmod(fake, 0755)
|
||||
f.install(t, fake)
|
||||
f.commit(t, "source.txt", []byte("clean\n"), "clean")
|
||||
f.push(t, 2, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("inline-and-config-suppression-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
f.commit(t, ".gitleaks.toml", []byte("[allowlist]\nregexes = ['.*']\n"), "config")
|
||||
f.commit(t, "secret.txt", append([]byte("# gitleaks:allow\n"), key(t)...), "fixture")
|
||||
f.push(t, 1, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("replacement-objects-cannot-hide-secret", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
clean := strings.TrimSpace(string(run(t, f.client, nil, "git", "rev-parse", "HEAD")))
|
||||
bad := f.commit(t, "secret.txt", key(t), "fixture")
|
||||
run(t, f.server, nil, "git", "fetch", "-q", f.client, "HEAD")
|
||||
run(t, f.server, nil, "git", "replace", bad, clean)
|
||||
f.push(t, 1, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("grafts-cannot-hide-introduced-history", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
f.commit(t, "secret.txt", key(t), "fixture")
|
||||
run(t, f.client, nil, "git", "rm", "-q", "secret.txt")
|
||||
run(t, f.client, nil, "git", "commit", "-qm", "remove")
|
||||
tip := strings.TrimSpace(string(run(t, f.client, nil, "git", "rev-parse", "HEAD")))
|
||||
write(t, filepath.Join(f.server, "info/grafts"), []byte(tip+"\n"))
|
||||
f.push(t, 1, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("ref-name-token", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
random := make([]byte, 10)
|
||||
rand.Read(random)
|
||||
f.push(t, 1, "HEAD:refs/heads/glpat-"+hex.EncodeToString(random))
|
||||
})
|
||||
t.Run("partial-config-include-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
config := filepath.Join(t.TempDir(), "partial-config")
|
||||
write(t, config, []byte("[remote \"origin\"]\n promisor = true\n"))
|
||||
run(t, f.server, nil, "git", "config", "include.path", config)
|
||||
f.commit(t, "source.txt", []byte("clean\n"), "clean")
|
||||
f.push(t, 2, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("object-count-limit-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
for i := 0; i < maxObjects; i++ {
|
||||
write(t, filepath.Join(f.client, fmt.Sprintf("file-%d.txt", i)), []byte(fmt.Sprintf("unique clean %d\n", i)))
|
||||
}
|
||||
run(t, f.client, nil, "git", "add", ".")
|
||||
run(t, f.client, nil, "git", "commit", "-qm", "large object set")
|
||||
f.push(t, 2, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("external-gitlink-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
head := strings.TrimSpace(string(run(t, f.client, nil, "git", "rev-parse", "HEAD")))
|
||||
run(t, f.client, nil, "git", "update-index", "--add", "--cacheinfo", "160000,"+head+",external")
|
||||
run(t, f.client, nil, "git", "commit", "-qm", "external gitlink")
|
||||
f.push(t, 2, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("partial-repository-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
run(t, f.server, nil, "git", "config", "remote.origin.promisor", "true")
|
||||
f.commit(t, "source.txt", []byte("clean\n"), "clean")
|
||||
f.push(t, 2, "HEAD:refs/heads/main")
|
||||
})
|
||||
t.Run("aggregate-byte-limit-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
for i := 0; i < 9; i++ {
|
||||
line := []byte(fmt.Sprintf("clean input %d\n", i))
|
||||
data := bytes.Repeat(line, (maxObject-1024)/len(line))
|
||||
write(t, filepath.Join(f.client, fmt.Sprintf("large-%d.txt", i)), data)
|
||||
}
|
||||
run(t, f.client, nil, "git", "add", ".")
|
||||
run(t, f.client, nil, "git", "commit", "-qm", "large aggregate")
|
||||
out := f.push(t, 2, "HEAD:refs/heads/main")
|
||||
if !bytes.Contains(out, []byte(`"reason":"object-byte-limit"`)) || !bytes.Contains(out, []byte(`"scanned_objects":0`)) {
|
||||
t.Fatal("aggregate bytes not bounded")
|
||||
}
|
||||
})
|
||||
t.Run("oversized-object-refused", func(t *testing.T) {
|
||||
f := setup(t, adapter, scanner)
|
||||
f.commit(t, "large.txt", bytes.Repeat([]byte("x"), maxObject+1), "large")
|
||||
f.push(t, 2, "HEAD:refs/heads/main")
|
||||
})
|
||||
}
|
||||
func TestScannerFileAndTimeoutRefusal(t *testing.T) {
|
||||
adapter, scanner := binary(t)
|
||||
f := setup(t, adapter, scanner)
|
||||
t.Run("nonexecutable", func(t *testing.T) {
|
||||
p := filepath.Join(t.TempDir(), "scanner")
|
||||
b, _ := os.ReadFile(scanner)
|
||||
write(t, p, b)
|
||||
if file, err := pinScanner(p); err == nil {
|
||||
file.Close()
|
||||
t.Fatal("accepted")
|
||||
}
|
||||
})
|
||||
t.Run("symlink", func(t *testing.T) {
|
||||
p := filepath.Join(t.TempDir(), "scanner")
|
||||
os.Symlink(scanner, p)
|
||||
if file, err := pinScanner(p); err == nil {
|
||||
file.Close()
|
||||
t.Fatal("accepted")
|
||||
}
|
||||
})
|
||||
t.Run("expired-context", func(t *testing.T) {
|
||||
old, _ := os.Getwd()
|
||||
os.Chdir(f.server)
|
||||
defer os.Chdir(old)
|
||||
before := strings.TrimSpace(string(run(t, f.server, nil, "git", "rev-parse", "refs/heads/main")))
|
||||
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(-time.Second))
|
||||
defer cancel()
|
||||
got := check(ctx, []update{{before, before, "refs/heads/main"}}, scanner)
|
||||
if got.Status != "unverified" {
|
||||
t.Fatal("timeout accepted")
|
||||
}
|
||||
})
|
||||
t.Run("scanner-process-failure", func(t *testing.T) {
|
||||
pinned, err := pinScanner(scanner)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pinned.Close()
|
||||
_, err = scan(context.Background(), pinned, t.TempDir(), []byte("clean"))
|
||||
if err == nil || err.Error() != "scanner-failed" {
|
||||
t.Fatal("scanner failure accepted")
|
||||
}
|
||||
})
|
||||
t.Run("unknown-old-ref", func(t *testing.T) {
|
||||
cmd := exec.Command(adapter, "--scanner", scanner)
|
||||
cmd.Dir = f.server
|
||||
cmd.Stdin = strings.NewReader(strings.Repeat("b", 40) + " " + strings.Repeat("a", 40) + " refs/heads/main\n")
|
||||
raw, err := cmd.Output()
|
||||
if err == nil {
|
||||
t.Fatal("accepted mismatch")
|
||||
}
|
||||
var got result
|
||||
if json.Unmarshal(raw, &got) != nil || got.Reason != "old-ref-mismatch" {
|
||||
t.Fatal("wrong refusal")
|
||||
}
|
||||
})
|
||||
}
|
||||
86
workplans/FORGE-WP-0006-independent-secret-detection.md
Normal file
86
workplans/FORGE-WP-0006-independent-secret-detection.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
---
|
||||
id: FORGE-WP-0006
|
||||
type: workplan
|
||||
title: "Enforce secret detection independently in the forge receive path"
|
||||
domain: financials
|
||||
repo: railiance-forge
|
||||
status: active
|
||||
owner: codex
|
||||
topic_slug: railiance
|
||||
created: "2026-10-12"
|
||||
updated: "2026-10-12"
|
||||
flavor: implementation
|
||||
depends_on: []
|
||||
---
|
||||
|
||||
Own the native forge boundary of RMGR-WP-0021-T03 / CUST-WP-0080 G07.
|
||||
Reuse the existing local scanner admission; do not duplicate incident custody,
|
||||
source-workflow ownership or founder-deferred rotation. The prior runner/resource
|
||||
workplans are finished and do not cover this missing control. This plan is
|
||||
source/preparation authorization, not a claim of independent owner assent or
|
||||
native installation.
|
||||
|
||||
## Implement introduced-history refusal and verify the protocol
|
||||
|
||||
```task
|
||||
id: FORGE-WP-0006-T01
|
||||
status: done
|
||||
priority: high
|
||||
```
|
||||
|
||||
Native baseline: railiance01 Forgejo 11.0.3 stores 165 repository directories;
|
||||
all 22 reviewed repositories have only upstream-generated receive hooks. Evidence:
|
||||
Custodian `docs/evidence/2026-10-12-netkingdom-security/forge-secret-detection-baseline.json`.
|
||||
The standalone adapter must cover all introduced reachable objects and ref names,
|
||||
intermediate/deleted secrets, multiple parents, messages/tags, quarantine, force
|
||||
updates, multi-ref refusal and scanner/tool failures. Preserve generated Forgejo
|
||||
hooks and suppress finding contents. Record real push tests and precise limits.
|
||||
|
||||
2026-10-12 source acceptance: the Linux Go adapter passes 33 leaf cases
|
||||
(36 passing test nodes), no skips, using pinned Gitleaks and verified official
|
||||
Go 1.26.9. Actual disposable Git pushes cover initial/SHA-256 repositories, clean/new/deleted branches, secret
|
||||
blobs, added-then-deleted history, second-parent merges, commit/tag messages,
|
||||
direct blob tags, force/multi-ref pushes, replacement/graft bypass refusal,
|
||||
ref-name detections, config/inline suppression refusal, missing/wrong scanner,
|
||||
partial clones (including included config), gitlinks and object count/byte limits.
|
||||
Batch bytes are refused before scanning payloads.
|
||||
Rejected quarantined commits were absent from the server object store. Scanner
|
||||
mode/symlink/process failures, timeout and invalid old-ref tests also pass. These
|
||||
prove adapter/protocol behavior only; no production hook is installed by T01.
|
||||
Initial test-build setup and unsupported Git flag failures were corrected and
|
||||
retained in Custodian's receipt; a rejected push without the expected detection
|
||||
status does not count as a successful secret-detection test.
|
||||
|
||||
## Admit the inventory and activate a reviewed server profile
|
||||
|
||||
```task
|
||||
id: FORGE-WP-0006-T02
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Classify the production-software denominator from the current forge metadata;
|
||||
165 directories are not sufficient. Admit exact scanner/adapter build and runtime
|
||||
versions, executable custody, bounded resource cost, reviewed hook composition,
|
||||
abort owner, backup/rollback and receipt. Start with an explicit reviewed set,
|
||||
record uncovered scope, and reconcile new repositories/mirrors/wikis/archives.
|
||||
Do not enable user-authored arbitrary hooks. Prove clean acceptance, secret and
|
||||
unavailable-scanner refusal, quarantine cleanup, unchanged authorization and
|
||||
other hook behavior. Native acceptance must include SSH/HTTP/frontend creation
|
||||
and merge paths plus restart/hook regeneration. Keep a missing proof open.
|
||||
|
||||
## Verify CI independence, detection response and complete coverage
|
||||
|
||||
```task
|
||||
id: FORGE-WP-0006-T03
|
||||
status: todo
|
||||
priority: high
|
||||
```
|
||||
|
||||
Coordinate existing RMGR-WP-0021-T03/T04 without inventing owner acceptance.
|
||||
Review who can alter workflows, required checks and enforcement. Exercise private
|
||||
synthetic finding classification/routing with a revocation/rotation owner, false
|
||||
positive governance, exact receipt and recovery path. Historical, log/artifact,
|
||||
runtime and LFS gaps stay explicit. Complete only with native prevention and
|
||||
response evidence for the admitted denominator; hand off residuals as live records
|
||||
before finishing. Do not claim all DSM-DETECT requirements from hook tests alone.
|
||||
Loading…
Add table
Add a link
Reference in a new issue