Implement Development Effort Calculator (WP-0010-T02)
src/target_revenue/effort_calculator.py implements Candidate A (labor-cost-anchored, accepted T01): commit-timestamp session-gap clustering for human interaction time, workplan/task-volume counts via direct workplans/ directory parsing (no state-hub dependency, works uniformly on any repo using this repo's own convention), file/line counts with generated/vendored-path exclusion, and caller-supplied token-cost pricing. estimate_target_basis() combines these and returns a derivation dict (every input shown) plus a warnings list - never a black-box dollar figure. 1-day manual-work floor, as requested: any raw commit-clustered estimate below 1.0 day is floored and flagged with a warning that this is very likely a measurement gap (commit-clustering is a floor estimate by design) that should usually be compensated for by manual override, not trusted at face value. A second, independent sanity-check warning fires when finished-workplan/task volume is substantial but the time estimate is still low, even above the floor - demonstrated live against target-revenue's own history (7 finished workplans, 57 tasks correctly flagged a 2.38-day estimate as under-counted). scripts/effort_calculator_cli.py: CLI wrapper printing JSON, following the same offline-first, no-Phase-declaration pattern as scripts/trf_onboard.py. tests/test_effort_calculator.py (15 deterministic tests, throwaway git repos/tmp_path fixtures) covers commit clustering, workplan/task parsing, size-metric exclusion, token-cost pricing, the floor-and-warning behavior, the sanity-check warning, and an end-to-end smoke test. No new hard dependency.
This commit is contained in:
parent
37ccee22ab
commit
9566e16a97
6 changed files with 720 additions and 2 deletions
|
|
@ -82,7 +82,7 @@ The concept's §13 now defines a **Global Contingency Share Determination Rule**
|
|||
| [TREV-WP-0007](workplans/TREV-WP-0007-degeneration-policy-and-canonical-profiles.md) | Degeneration policy + canonical monetization profile catalog — **finished**, all 4 tasks done. `trsl:policy:linear-longstop-v0` confirmed 2026-07-29 as the v1 norm for the first pilot cohort; `progress-paused-longstop-v1` named as the next iteration, not yet adopted |
|
||||
| [TREV-WP-0008](workplans/TREV-WP-0008-governance-and-pilot-rollout.md) | Governance formalization + pilot rollout — active; T01–T04 done. `info-tech-canon` dry-run onboarding routine exercised end-to-end 2026-07-29. **Org-wide TRSL license adoption executed 2026-07-30** across ~90 `coulomb`-org repos (`history/260730-TRSL-OrgWideLicenseRollout.md`) — a license-text adoption, not a Phase declaration. T05 (real Phase go-live gate) remains `todo` by design; no Phase exists yet for any repo |
|
||||
| [TREV-WP-0009](workplans/TREV-WP-0009-target-revenue-control-plane.md) | Target Revenue Control Plane — interactive UI for the `binky` tenant, incl. interactive Development Credit entry creation (`specs/TargetRevenueControlPlaneConcept.md`) — active; **T01 accepted 2026-07-30** (four rights tiers confirmed; per-human sub-credentials at the Trust Service layer chosen over the concept's own simpler recommendation) — adds a real prerequisite: T02 extends WP-0006's finished auth layer before T03 (Control Plane backend) and T04 (interactive UI) can proceed |
|
||||
| [TREV-WP-0010](workplans/TREV-WP-0010-development-effort-calculator.md) | Development Effort Calculator — turns human time/workplan-task volume/repo size/AI token cost into `target_basis` values feeding the framework's existing Initial Target formula (`specs/DevelopmentEffortCalculatorConcept.md`) — active; **T01 accepted 2026-07-30 (Candidate A, labor-cost-anchored)**; T02 (implementation) next. Split from TREV-WP-0009 2026-07-30 |
|
||||
| [TREV-WP-0010](workplans/TREV-WP-0010-development-effort-calculator.md) | Development Effort Calculator — turns human time/workplan-task volume/repo size/AI token cost into `target_basis` values feeding the framework's existing Initial Target formula (`specs/DevelopmentEffortCalculatorConcept.md`) — active; T01 (Candidate A) and **T02 (implementation, `src/target_revenue/effort_calculator.py`, incl. a 1-day manual-work floor with warning) done**; T03 (apply to real candidates) next |
|
||||
|
||||
Hub index: [`WORK-RECORDS.md`](WORK-RECORDS.md) · brief: [`.custodian-brief.md`](.custodian-brief.md)
|
||||
|
||||
|
|
|
|||
57
scripts/effort_calculator_cli.py
Normal file
57
scripts/effort_calculator_cli.py
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
#!/usr/bin/env python3
|
||||
"""CLI wrapper for target_revenue.effort_calculator (WP-0010-T02).
|
||||
|
||||
Prints a proposed `phase.target_basis` estimate for a repo, with full
|
||||
derivation and warnings, as JSON. Token counts are caller-supplied (e.g.
|
||||
pulled from the state hub's `get_token_summary` beforehand) rather than
|
||||
fetched here — this script has no MCP client of its own, matching the
|
||||
offline-first design of the rest of this library.
|
||||
|
||||
Does not select a Target Multiple and does not register anything with
|
||||
the hosted Trust Service — this is an estimation aid, not a Phase
|
||||
declaration tool (that remains `scripts/trf_onboard.py`).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import dataclasses
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO_ROOT / "src"))
|
||||
|
||||
from target_revenue import effort_calculator as ec # noqa: E402
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("repo_path", help="path to the repo to estimate effort for")
|
||||
parser.add_argument("--daily-rate", type=float, required=True)
|
||||
parser.add_argument("--tokens-in", type=int, default=0)
|
||||
parser.add_argument("--tokens-out", type=int, default=0)
|
||||
parser.add_argument("--model", default="default")
|
||||
parser.add_argument("--session-gap-hours", type=float, default=ec.DEFAULT_SESSION_GAP_HOURS)
|
||||
parser.add_argument("--hours-per-day", type=float, default=ec.DEFAULT_HOURS_PER_DAY)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
result = ec.calculate_target_basis(
|
||||
args.repo_path,
|
||||
daily_rate=args.daily_rate,
|
||||
tokens_in=args.tokens_in,
|
||||
tokens_out=args.tokens_out,
|
||||
model=args.model,
|
||||
session_gap_hours=args.session_gap_hours,
|
||||
hours_per_day=args.hours_per_day,
|
||||
)
|
||||
|
||||
output = dataclasses.asdict(result)
|
||||
print(json.dumps(output, indent=2))
|
||||
if result.warnings:
|
||||
print(f"\n{len(result.warnings)} warning(s) — see 'warnings' above.", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -67,6 +67,18 @@ across every repo without depending on hub coverage — explicitly a
|
|||
*floor* estimate (real effort is almost certainly higher), not a
|
||||
best-effort true count.
|
||||
|
||||
**Implemented 2026-07-30** (`workplans/TREV-WP-0010-development-effort-calculator.md`
|
||||
T02, `src/target_revenue/effort_calculator.py`), including a maintainer-
|
||||
requested **1-day manual-work floor**: any raw commit-clustered estimate
|
||||
below one day is floored to one day and flagged with a warning explaining
|
||||
this is very likely a measurement gap, not a true "under a day" fact —
|
||||
and should usually be compensated for by a manual override of
|
||||
`estimated_effort_days` rather than trusted at face value. A second,
|
||||
independent sanity-check warning fires whenever a repo's finished-
|
||||
workplan/task volume (§2b) is substantial but the raw time estimate is
|
||||
still low, catching cases above the 1-day floor that still look
|
||||
implausible.
|
||||
|
||||
### 2b. Workplan/task volume — complexity index component
|
||||
|
||||
**What:** number and status distribution of workplans and tasks
|
||||
|
|
|
|||
361
src/target_revenue/effort_calculator.py
Normal file
361
src/target_revenue/effort_calculator.py
Normal file
|
|
@ -0,0 +1,361 @@
|
|||
"""Development Effort Calculator (WP-0010-T02).
|
||||
|
||||
Implements Candidate A (labor-cost-anchored), accepted 2026-07-30
|
||||
(`workplans/TREV-WP-0010-development-effort-calculator.md` T01,
|
||||
`specs/DevelopmentEffortCalculatorConcept.md` §3): human interaction time
|
||||
drives `estimated_effort_days`/`daily_rate` directly; AI token cost
|
||||
becomes `approved_direct_costs`; workplan/task-volume and repo-size
|
||||
metrics are used only as a sanity check on the human-time estimate, never
|
||||
their own dollar figure. This module produces `target_basis` values
|
||||
(`specs/PhaseManifestSpecification.md`) feeding the framework's existing
|
||||
Initial Target formula — it does not compute an Initial Target itself,
|
||||
and it never chooses a Target Multiple (that remains a human judgment
|
||||
call, concept §4).
|
||||
|
||||
Every function here is pure or reads only from the local filesystem/git
|
||||
history — no network calls, no dependency on the state hub being
|
||||
reachable, consistent with the offline-first design already established
|
||||
elsewhere in this library (`fold.py`, `validation.py`).
|
||||
|
||||
**1-day manual-work floor (explicit, not silent):** commit-timestamp
|
||||
clustering is, by the concept document's own framing, a floor estimate —
|
||||
it systematically undercounts real effort (thinking/reading time, work
|
||||
that never produced a commit). Any raw estimate below one day is treated
|
||||
as a signal the measurement itself is incomplete, not as a true "this took
|
||||
under a day" fact. `estimate_target_basis` floors the reported value to
|
||||
`MANUAL_EFFORT_FLOOR_DAYS` in that case and attaches a prominent warning
|
||||
recommending a manual override — the floor is a safe fallback so the
|
||||
Phase Manifest never carries an implausibly small figure, not a
|
||||
replacement for a human's own judgment about the real effort involved.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
MANUAL_EFFORT_FLOOR_DAYS = 1.0
|
||||
DEFAULT_SESSION_GAP_HOURS = 2.0
|
||||
DEFAULT_HOURS_PER_DAY = 8.0
|
||||
SINGLE_COMMIT_SESSION_MINUTES = 15.0
|
||||
|
||||
# v0 placeholder pricing — flagged for refinement once real usage data
|
||||
# exists; not authoritative, callers may override via `model_pricing`.
|
||||
DEFAULT_MODEL_PRICING_USD_PER_TOKEN: dict[str, dict[str, float]] = {
|
||||
"default": {"input": 3e-6, "output": 15e-6},
|
||||
}
|
||||
|
||||
# Paths that inflate the repo-size signal without reflecting real
|
||||
# authored effort (specs/DevelopmentEffortCalculatorConcept.md §2c's
|
||||
# flagged distortion). Matched against any path component.
|
||||
GENERATED_VENDORED_EXCLUDES = frozenset(
|
||||
{
|
||||
"node_modules", ".venv", "venv", "__pycache__", ".git",
|
||||
"dist", "build", ".pytest_cache", "vendor", "site-packages",
|
||||
".egg-info", ".mypy_cache", ".ruff_cache",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CommitTimeEstimate:
|
||||
"""Result of session-gap clustering over a repo's commit timestamps."""
|
||||
|
||||
raw_hours: float
|
||||
session_count: int
|
||||
commit_count: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WorkplanTaskCounts:
|
||||
"""Workplan/task volume, used only as a sanity-check signal (§2b)."""
|
||||
|
||||
finished_workplans: int
|
||||
total_tasks: int
|
||||
finished_tasks: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RepoSizeMetrics:
|
||||
"""File/line counts, used only as a sanity-check signal (§2c)."""
|
||||
|
||||
file_count: int
|
||||
line_count: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TokenCost:
|
||||
"""AI token cost — the strongest, least speculative input (§2d)."""
|
||||
|
||||
tokens_in: int
|
||||
tokens_out: int
|
||||
usd: float
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TargetBasisEstimate:
|
||||
"""Proposed `phase.target_basis` values, with visible derivation.
|
||||
|
||||
`warnings` and `derivation` exist so the output is never a black-box
|
||||
dollar figure — per `specs/TargetRevenueLicenseConcept.md` §4.6's
|
||||
"transparent, non-gameable" goal, showing which inputs produced which
|
||||
numbers is part of this calculator's job, not an optional extra.
|
||||
"""
|
||||
|
||||
estimated_effort_days: float
|
||||
daily_rate: float
|
||||
approved_direct_costs: float
|
||||
warnings: list[str] = field(default_factory=list)
|
||||
derivation: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
def cluster_commit_hours(
|
||||
repo_path: str | Path, session_gap_hours: float = DEFAULT_SESSION_GAP_HOURS
|
||||
) -> CommitTimeEstimate:
|
||||
"""Group commit timestamps into sessions (gap-based clustering).
|
||||
|
||||
A session is a run of commits with no gap larger than
|
||||
`session_gap_hours` between consecutive commits. Each session
|
||||
contributes its own wall-clock span; a session of a single commit
|
||||
(span zero) contributes a fixed `SINGLE_COMMIT_SESSION_MINUTES` floor,
|
||||
since a lone commit still took some real, non-zero time to produce.
|
||||
|
||||
This is an explicit floor estimate (concept §2a), not a true count —
|
||||
time spent reading, thinking, or working without committing is
|
||||
invisible to it by construction.
|
||||
"""
|
||||
result = subprocess.run(
|
||||
["git", "-C", str(repo_path), "log", "--all", "--format=%at"],
|
||||
capture_output=True, text=True, check=True,
|
||||
)
|
||||
timestamps = sorted(int(line) for line in result.stdout.splitlines() if line.strip())
|
||||
if not timestamps:
|
||||
return CommitTimeEstimate(raw_hours=0.0, session_count=0, commit_count=0)
|
||||
|
||||
gap_seconds = session_gap_hours * 3600
|
||||
sessions: list[tuple[int, int]] = []
|
||||
session_start = timestamps[0]
|
||||
session_end = timestamps[0]
|
||||
for ts in timestamps[1:]:
|
||||
if ts - session_end > gap_seconds:
|
||||
sessions.append((session_start, session_end))
|
||||
session_start = ts
|
||||
session_end = ts
|
||||
sessions.append((session_start, session_end))
|
||||
|
||||
total_hours = 0.0
|
||||
for start, end in sessions:
|
||||
span_hours = (end - start) / 3600
|
||||
if span_hours <= 0:
|
||||
span_hours = SINGLE_COMMIT_SESSION_MINUTES / 60
|
||||
total_hours += span_hours
|
||||
|
||||
return CommitTimeEstimate(
|
||||
raw_hours=total_hours, session_count=len(sessions), commit_count=len(timestamps)
|
||||
)
|
||||
|
||||
|
||||
_TASK_BLOCK_RE = re.compile(r"```task\n(.*?)\n```", re.DOTALL)
|
||||
_FRONTMATTER_RE = re.compile(r"^---\n(.*?)\n---", re.DOTALL)
|
||||
_STATUS_FIELD_RE = re.compile(r'^status:\s*"?([^"\n]+)"?\s*$', re.MULTILINE)
|
||||
|
||||
|
||||
def workplan_task_counts(repo_path: str | Path) -> WorkplanTaskCounts:
|
||||
"""Count finished workplans and tasks under `<repo>/workplans/`.
|
||||
|
||||
Follows this repo's own workplan convention (YAML-ish frontmatter with
|
||||
a `status:` field, ```task fenced blocks each with their own
|
||||
`status:` line) rather than requiring state-hub connectivity — works
|
||||
on any repo using this convention, degrades to zero (not an error) for
|
||||
repos with no `workplans/` directory at all.
|
||||
"""
|
||||
workplans_dir = Path(repo_path) / "workplans"
|
||||
if not workplans_dir.is_dir():
|
||||
return WorkplanTaskCounts(finished_workplans=0, total_tasks=0, finished_tasks=0)
|
||||
|
||||
finished_workplans = 0
|
||||
total_tasks = 0
|
||||
finished_tasks = 0
|
||||
|
||||
for md_file in sorted(workplans_dir.glob("*.md")):
|
||||
try:
|
||||
text = md_file.read_text(encoding="utf-8", errors="ignore")
|
||||
except OSError:
|
||||
continue
|
||||
|
||||
frontmatter_match = _FRONTMATTER_RE.match(text)
|
||||
if frontmatter_match:
|
||||
status_match = _STATUS_FIELD_RE.search(frontmatter_match.group(1))
|
||||
if status_match and status_match.group(1).strip() == "finished":
|
||||
finished_workplans += 1
|
||||
|
||||
for block in _TASK_BLOCK_RE.findall(text):
|
||||
total_tasks += 1
|
||||
task_status_match = _STATUS_FIELD_RE.search(block)
|
||||
if task_status_match and task_status_match.group(1).strip() == "done":
|
||||
finished_tasks += 1
|
||||
|
||||
return WorkplanTaskCounts(
|
||||
finished_workplans=finished_workplans,
|
||||
total_tasks=total_tasks,
|
||||
finished_tasks=finished_tasks,
|
||||
)
|
||||
|
||||
|
||||
def repo_size_metrics(
|
||||
repo_path: str | Path, extra_excludes: frozenset[str] = frozenset()
|
||||
) -> RepoSizeMetrics:
|
||||
"""File/line counts, excluding generated/vendored/dependency paths.
|
||||
|
||||
Exclusion is by path-component membership (any directory named
|
||||
`node_modules`, `.venv`, etc. anywhere in a file's path is skipped
|
||||
entirely), not just top-level — this catches nested vendored trees,
|
||||
not only ones at the repo root.
|
||||
"""
|
||||
excludes = GENERATED_VENDORED_EXCLUDES | extra_excludes
|
||||
file_count = 0
|
||||
line_count = 0
|
||||
for path in Path(repo_path).rglob("*"):
|
||||
if not path.is_file():
|
||||
continue
|
||||
if any(part in excludes for part in path.parts):
|
||||
continue
|
||||
file_count += 1
|
||||
try:
|
||||
with open(path, "rb") as f:
|
||||
line_count += sum(1 for _ in f)
|
||||
except OSError:
|
||||
continue
|
||||
return RepoSizeMetrics(file_count=file_count, line_count=line_count)
|
||||
|
||||
|
||||
def token_cost_usd(
|
||||
tokens_in: int,
|
||||
tokens_out: int,
|
||||
model: str = "default",
|
||||
model_pricing: dict[str, dict[str, float]] | None = None,
|
||||
) -> float:
|
||||
"""AI token cost in USD for the given token counts and model.
|
||||
|
||||
`model_pricing` defaults to `DEFAULT_MODEL_PRICING_USD_PER_TOKEN` — a
|
||||
v0 placeholder table, not authoritative pricing; callers with real
|
||||
per-model rates should pass their own table. Unknown models fall back
|
||||
to `"default"` rather than raising, since the exact model used is a
|
||||
`get_token_summary` reporting detail this module has no way to
|
||||
validate against a live price list.
|
||||
"""
|
||||
pricing_table = model_pricing or DEFAULT_MODEL_PRICING_USD_PER_TOKEN
|
||||
pricing = pricing_table.get(model, pricing_table["default"])
|
||||
return tokens_in * pricing["input"] + tokens_out * pricing["output"]
|
||||
|
||||
|
||||
def estimate_target_basis(
|
||||
*,
|
||||
commit_time: CommitTimeEstimate,
|
||||
workplan_tasks: WorkplanTaskCounts,
|
||||
repo_size: RepoSizeMetrics,
|
||||
token_cost: TokenCost,
|
||||
daily_rate: float,
|
||||
hours_per_day: float = DEFAULT_HOURS_PER_DAY,
|
||||
) -> TargetBasisEstimate:
|
||||
"""Combine the four metric families into proposed `target_basis` values.
|
||||
|
||||
Candidate A (labor-cost-anchored): `estimated_effort_days` comes
|
||||
directly from `commit_time`; `approved_direct_costs` comes directly
|
||||
from `token_cost`; `workplan_tasks`/`repo_size` never produce their
|
||||
own dollar figure — they only trigger sanity-check warnings when they
|
||||
suggest the time estimate is implausibly low for the amount of
|
||||
finished, real work on record.
|
||||
"""
|
||||
warnings: list[str] = []
|
||||
derivation: dict[str, Any] = {}
|
||||
|
||||
raw_effort_days = commit_time.raw_hours / hours_per_day
|
||||
derivation["raw_commit_clustered_hours"] = round(commit_time.raw_hours, 3)
|
||||
derivation["raw_commit_clustered_days"] = round(raw_effort_days, 3)
|
||||
derivation["sessions"] = commit_time.session_count
|
||||
derivation["commits"] = commit_time.commit_count
|
||||
|
||||
complexity_units_workplans = (
|
||||
workplan_tasks.finished_workplans * 3 + workplan_tasks.finished_tasks * 1
|
||||
)
|
||||
derivation["finished_workplans"] = workplan_tasks.finished_workplans
|
||||
derivation["finished_tasks"] = workplan_tasks.finished_tasks
|
||||
derivation["complexity_units_workplans"] = complexity_units_workplans
|
||||
derivation["file_count"] = repo_size.file_count
|
||||
derivation["line_count"] = repo_size.line_count
|
||||
|
||||
effort_days = raw_effort_days
|
||||
if effort_days < MANUAL_EFFORT_FLOOR_DAYS:
|
||||
warnings.append(
|
||||
f"Raw commit-clustered estimate was {raw_effort_days:.3f} day(s), below the "
|
||||
f"{MANUAL_EFFORT_FLOOR_DAYS:.1f}-day floor. Floored to "
|
||||
f"{MANUAL_EFFORT_FLOOR_DAYS:.1f} day(s) rather than reporting an implausibly "
|
||||
"small figure. This is very likely a measurement gap — commit-timestamp "
|
||||
"clustering is a floor estimate by design "
|
||||
"(specs/DevelopmentEffortCalculatorConcept.md §2a), not a true reflection of "
|
||||
"effort — and should usually be compensated for by a manual override of "
|
||||
"estimated_effort_days rather than trusting this floored value at face value."
|
||||
)
|
||||
effort_days = MANUAL_EFFORT_FLOOR_DAYS
|
||||
|
||||
# Sanity-check signal (§2b/§2c): a real, non-trivial finished-work
|
||||
# history alongside a still-low time estimate is itself evidence the
|
||||
# time input is under-counting, independent of the floor above.
|
||||
if complexity_units_workplans >= 15 and raw_effort_days < MANUAL_EFFORT_FLOOR_DAYS * 3:
|
||||
warnings.append(
|
||||
f"{workplan_tasks.finished_workplans} finished workplan(s) and "
|
||||
f"{workplan_tasks.finished_tasks} finished task(s) on record suggest more real "
|
||||
f"effort than the {raw_effort_days:.2f}-day raw time estimate reflects — treat "
|
||||
"the time estimate as under-counted and consider a manual override."
|
||||
)
|
||||
|
||||
derivation["approved_direct_costs_usd"] = round(token_cost.usd, 2)
|
||||
derivation["tokens_in"] = token_cost.tokens_in
|
||||
derivation["tokens_out"] = token_cost.tokens_out
|
||||
|
||||
return TargetBasisEstimate(
|
||||
estimated_effort_days=round(effort_days, 3),
|
||||
daily_rate=daily_rate,
|
||||
approved_direct_costs=round(token_cost.usd, 2),
|
||||
warnings=warnings,
|
||||
derivation=derivation,
|
||||
)
|
||||
|
||||
|
||||
def calculate_target_basis(
|
||||
repo_path: str | Path,
|
||||
daily_rate: float,
|
||||
tokens_in: int = 0,
|
||||
tokens_out: int = 0,
|
||||
model: str = "default",
|
||||
session_gap_hours: float = DEFAULT_SESSION_GAP_HOURS,
|
||||
hours_per_day: float = DEFAULT_HOURS_PER_DAY,
|
||||
extra_excludes: frozenset[str] = frozenset(),
|
||||
model_pricing: dict[str, dict[str, float]] | None = None,
|
||||
) -> TargetBasisEstimate:
|
||||
"""End-to-end convenience entry point: repo path + token counts in,
|
||||
a proposed `target_basis` estimate (with warnings and derivation) out.
|
||||
|
||||
`tokens_in`/`tokens_out` are caller-supplied (e.g. from the state
|
||||
hub's `get_token_summary`) rather than fetched here, since this
|
||||
module has no MCP/network client of its own — keeping it offline-first
|
||||
like the rest of this library.
|
||||
"""
|
||||
commit_time = cluster_commit_hours(repo_path, session_gap_hours=session_gap_hours)
|
||||
workplan_tasks = workplan_task_counts(repo_path)
|
||||
repo_size = repo_size_metrics(repo_path, extra_excludes=extra_excludes)
|
||||
usd = token_cost_usd(tokens_in, tokens_out, model=model, model_pricing=model_pricing)
|
||||
token_cost = TokenCost(tokens_in=tokens_in, tokens_out=tokens_out, usd=usd)
|
||||
|
||||
return estimate_target_basis(
|
||||
commit_time=commit_time,
|
||||
workplan_tasks=workplan_tasks,
|
||||
repo_size=repo_size,
|
||||
token_cost=token_cost,
|
||||
daily_rate=daily_rate,
|
||||
hours_per_day=hours_per_day,
|
||||
)
|
||||
246
tests/test_effort_calculator.py
Normal file
246
tests/test_effort_calculator.py
Normal file
|
|
@ -0,0 +1,246 @@
|
|||
"""Tests for target_revenue.effort_calculator (WP-0010-T02).
|
||||
|
||||
Pure/offline: builds throwaway git repos and directory fixtures under
|
||||
tmp_path rather than depending on any real repo's current state, so
|
||||
these tests stay deterministic regardless of what target-revenue's own
|
||||
history looks like later.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from target_revenue import effort_calculator as ec
|
||||
|
||||
|
||||
def _git(repo: Path, *args: str, env: dict | None = None) -> None:
|
||||
subprocess.run(["git", "-C", str(repo), *args], check=True, capture_output=True, env=env)
|
||||
|
||||
|
||||
def _commit(repo: Path, message: str, epoch_seconds: int) -> None:
|
||||
(repo / "file.txt").write_text(message)
|
||||
_git(repo, "add", "file.txt")
|
||||
date = f"{epoch_seconds} +0000"
|
||||
env = {
|
||||
"GIT_AUTHOR_DATE": date,
|
||||
"GIT_COMMITTER_DATE": date,
|
||||
"GIT_AUTHOR_NAME": "test",
|
||||
"GIT_AUTHOR_EMAIL": "test@example.com",
|
||||
"GIT_COMMITTER_NAME": "test",
|
||||
"GIT_COMMITTER_EMAIL": "test@example.com",
|
||||
}
|
||||
import os
|
||||
|
||||
full_env = {**os.environ, **env}
|
||||
_git(repo, "commit", "-q", "-m", message, env=full_env)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def git_repo(tmp_path):
|
||||
repo = tmp_path / "repo"
|
||||
repo.mkdir()
|
||||
_git(repo, "init", "-q", "-b", "main")
|
||||
return repo
|
||||
|
||||
|
||||
# --- cluster_commit_hours ----------------------------------------------
|
||||
|
||||
|
||||
def test_cluster_commit_hours_empty_repo_no_commits(git_repo):
|
||||
result = ec.cluster_commit_hours(git_repo)
|
||||
assert result == ec.CommitTimeEstimate(raw_hours=0.0, session_count=0, commit_count=0)
|
||||
|
||||
|
||||
def test_cluster_commit_hours_single_commit_uses_floor(git_repo):
|
||||
_commit(git_repo, "init", 1_000_000_000)
|
||||
result = ec.cluster_commit_hours(git_repo)
|
||||
assert result.commit_count == 1
|
||||
assert result.session_count == 1
|
||||
assert result.raw_hours == pytest.approx(ec.SINGLE_COMMIT_SESSION_MINUTES / 60)
|
||||
|
||||
|
||||
def test_cluster_commit_hours_one_session_spans_actual_time(git_repo):
|
||||
base = 1_000_000_000
|
||||
_commit(git_repo, "c1", base)
|
||||
_commit(git_repo, "c2", base + 3600) # 1 hour later, same session
|
||||
result = ec.cluster_commit_hours(git_repo, session_gap_hours=2.0)
|
||||
assert result.session_count == 1
|
||||
assert result.commit_count == 2
|
||||
assert result.raw_hours == pytest.approx(1.0)
|
||||
|
||||
|
||||
def test_cluster_commit_hours_gap_splits_into_two_sessions(git_repo):
|
||||
base = 1_000_000_000
|
||||
_commit(git_repo, "c1", base)
|
||||
_commit(git_repo, "c2", base + 3 * 3600) # 3 hours later, gap > 2h default
|
||||
result = ec.cluster_commit_hours(git_repo, session_gap_hours=2.0)
|
||||
assert result.session_count == 2
|
||||
assert result.commit_count == 2
|
||||
# Both are single-commit sessions -> two floor contributions.
|
||||
assert result.raw_hours == pytest.approx(2 * ec.SINGLE_COMMIT_SESSION_MINUTES / 60)
|
||||
|
||||
|
||||
# --- workplan_task_counts ------------------------------------------------
|
||||
|
||||
|
||||
def test_workplan_task_counts_no_workplans_dir(tmp_path):
|
||||
result = ec.workplan_task_counts(tmp_path)
|
||||
assert result == ec.WorkplanTaskCounts(0, 0, 0)
|
||||
|
||||
|
||||
def test_workplan_task_counts_counts_finished_workplans_and_tasks(tmp_path):
|
||||
workplans = tmp_path / "workplans"
|
||||
workplans.mkdir()
|
||||
(workplans / "WP-0001.md").write_text(
|
||||
"""---
|
||||
id: WP-0001
|
||||
status: finished
|
||||
---
|
||||
|
||||
```task
|
||||
id: WP-0001-T01
|
||||
status: done
|
||||
```
|
||||
|
||||
```task
|
||||
id: WP-0001-T02
|
||||
status: todo
|
||||
```
|
||||
"""
|
||||
)
|
||||
(workplans / "WP-0002.md").write_text(
|
||||
"""---
|
||||
id: WP-0002
|
||||
status: active
|
||||
---
|
||||
|
||||
```task
|
||||
id: WP-0002-T01
|
||||
status: done
|
||||
```
|
||||
"""
|
||||
)
|
||||
result = ec.workplan_task_counts(tmp_path)
|
||||
assert result.finished_workplans == 1
|
||||
assert result.total_tasks == 3
|
||||
assert result.finished_tasks == 2
|
||||
|
||||
|
||||
# --- repo_size_metrics ----------------------------------------------------
|
||||
|
||||
|
||||
def test_repo_size_metrics_counts_files_and_lines(tmp_path):
|
||||
(tmp_path / "a.py").write_text("line1\nline2\nline3\n")
|
||||
(tmp_path / "b.py").write_text("line1\n")
|
||||
result = ec.repo_size_metrics(tmp_path)
|
||||
assert result.file_count == 2
|
||||
assert result.line_count == 4
|
||||
|
||||
|
||||
def test_repo_size_metrics_excludes_vendored_paths(tmp_path):
|
||||
(tmp_path / "a.py").write_text("line1\n")
|
||||
vendored = tmp_path / "node_modules" / "pkg"
|
||||
vendored.mkdir(parents=True)
|
||||
(vendored / "big.js").write_text("x\n" * 1000)
|
||||
result = ec.repo_size_metrics(tmp_path)
|
||||
assert result.file_count == 1
|
||||
assert result.line_count == 1
|
||||
|
||||
|
||||
# --- token_cost_usd --------------------------------------------------------
|
||||
|
||||
|
||||
def test_token_cost_usd_default_pricing():
|
||||
cost = ec.token_cost_usd(1_000_000, 1_000_000)
|
||||
expected = 1_000_000 * 3e-6 + 1_000_000 * 15e-6
|
||||
assert cost == pytest.approx(expected)
|
||||
|
||||
|
||||
def test_token_cost_usd_unknown_model_falls_back_to_default():
|
||||
cost_unknown = ec.token_cost_usd(1000, 1000, model="some-unlisted-model")
|
||||
cost_default = ec.token_cost_usd(1000, 1000, model="default")
|
||||
assert cost_unknown == cost_default
|
||||
|
||||
|
||||
# --- estimate_target_basis: the 1-day floor and warning -------------------
|
||||
|
||||
|
||||
def test_low_raw_estimate_triggers_floor_and_warning():
|
||||
commit_time = ec.CommitTimeEstimate(raw_hours=0.5, session_count=1, commit_count=1)
|
||||
result = ec.estimate_target_basis(
|
||||
commit_time=commit_time,
|
||||
workplan_tasks=ec.WorkplanTaskCounts(0, 0, 0),
|
||||
repo_size=ec.RepoSizeMetrics(1, 10),
|
||||
token_cost=ec.TokenCost(0, 0, 0.0),
|
||||
daily_rate=1000.0,
|
||||
)
|
||||
assert result.estimated_effort_days == ec.MANUAL_EFFORT_FLOOR_DAYS
|
||||
assert any("floor" in w.lower() for w in result.warnings)
|
||||
assert any("manual override" in w.lower() for w in result.warnings)
|
||||
assert result.derivation["raw_commit_clustered_days"] < ec.MANUAL_EFFORT_FLOOR_DAYS
|
||||
|
||||
|
||||
def test_sufficient_raw_estimate_no_floor_warning():
|
||||
commit_time = ec.CommitTimeEstimate(
|
||||
raw_hours=ec.DEFAULT_HOURS_PER_DAY * 5, session_count=5, commit_count=20
|
||||
)
|
||||
result = ec.estimate_target_basis(
|
||||
commit_time=commit_time,
|
||||
workplan_tasks=ec.WorkplanTaskCounts(0, 0, 0),
|
||||
repo_size=ec.RepoSizeMetrics(10, 500),
|
||||
token_cost=ec.TokenCost(0, 0, 0.0),
|
||||
daily_rate=1000.0,
|
||||
)
|
||||
assert result.estimated_effort_days == pytest.approx(5.0)
|
||||
assert not any("floor" in w.lower() for w in result.warnings)
|
||||
|
||||
|
||||
def test_high_workplan_volume_with_low_time_triggers_sanity_warning():
|
||||
commit_time = ec.CommitTimeEstimate(raw_hours=4.0, session_count=1, commit_count=2)
|
||||
result = ec.estimate_target_basis(
|
||||
commit_time=commit_time,
|
||||
workplan_tasks=ec.WorkplanTaskCounts(finished_workplans=10, total_tasks=50, finished_tasks=40),
|
||||
repo_size=ec.RepoSizeMetrics(200, 20000),
|
||||
token_cost=ec.TokenCost(0, 0, 0.0),
|
||||
daily_rate=1000.0,
|
||||
)
|
||||
assert any("under-counted" in w.lower() for w in result.warnings)
|
||||
|
||||
|
||||
def test_derivation_shows_its_work():
|
||||
commit_time = ec.CommitTimeEstimate(raw_hours=16.0, session_count=2, commit_count=10)
|
||||
result = ec.estimate_target_basis(
|
||||
commit_time=commit_time,
|
||||
workplan_tasks=ec.WorkplanTaskCounts(2, 5, 3),
|
||||
repo_size=ec.RepoSizeMetrics(50, 3000),
|
||||
token_cost=ec.TokenCost(tokens_in=100, tokens_out=200, usd=1.23),
|
||||
daily_rate=800.0,
|
||||
)
|
||||
assert result.derivation["commits"] == 10
|
||||
assert result.derivation["sessions"] == 2
|
||||
assert result.derivation["finished_workplans"] == 2
|
||||
assert result.derivation["file_count"] == 50
|
||||
assert result.derivation["line_count"] == 3000
|
||||
assert result.derivation["tokens_in"] == 100
|
||||
assert result.approved_direct_costs == 1.23
|
||||
assert result.daily_rate == 800.0
|
||||
|
||||
|
||||
# --- calculate_target_basis: end-to-end -----------------------------------
|
||||
|
||||
|
||||
def test_calculate_target_basis_end_to_end(git_repo):
|
||||
base = 1_000_000_000
|
||||
_commit(git_repo, "c1", base)
|
||||
_commit(git_repo, "c2", base + ec.DEFAULT_HOURS_PER_DAY * 3 * 3600)
|
||||
result = ec.calculate_target_basis(
|
||||
git_repo, daily_rate=1000.0, tokens_in=1000, tokens_out=1000
|
||||
)
|
||||
assert result.daily_rate == 1000.0
|
||||
assert result.approved_direct_costs == pytest.approx(
|
||||
ec.token_cost_usd(1000, 1000), abs=0.01
|
||||
)
|
||||
assert isinstance(result.estimated_effort_days, float)
|
||||
|
|
@ -80,7 +80,7 @@ this formula, not Candidate B or a hybrid.
|
|||
|
||||
```task
|
||||
id: TREV-WP-0010-T02
|
||||
status: todo
|
||||
status: done
|
||||
priority: high
|
||||
state_hub_task_id: "b5fb46ea-b737-495f-82b8-5e10f0032cb5"
|
||||
```
|
||||
|
|
@ -107,6 +107,48 @@ black-box dollar figure with no visible derivation would undermine the
|
|||
"transparent, non-gameable" goal (`specs/TargetRevenueLicenseConcept.md`
|
||||
§4.6) this whole framework is built around.
|
||||
|
||||
**Result:** `src/target_revenue/effort_calculator.py` implemented, pure/
|
||||
offline (no network or state-hub dependency — `list_tasks`/`list_workplans`
|
||||
was superseded by direct `workplans/` directory parsing so the module
|
||||
works uniformly on any repo using this repo's own workplan convention,
|
||||
without requiring hub connectivity). `cluster_commit_hours()` does
|
||||
session-gap clustering over `git log --all --format=%at`, with a 15-minute
|
||||
floor for single-commit sessions. `workplan_task_counts()` parses
|
||||
frontmatter `status:` fields and this repo's triple-backtick task-block
|
||||
convention. `repo_size_metrics()`
|
||||
excludes generated/vendored path components
|
||||
(`node_modules`, `.venv`, `__pycache__`, etc.). `token_cost_usd()` takes
|
||||
caller-supplied token counts (e.g. from `get_token_summary`) rather than
|
||||
fetching them itself. `estimate_target_basis()` combines these per
|
||||
Candidate A and returns a `TargetBasisEstimate` with a `derivation` dict
|
||||
(showing every input value) and a `warnings` list.
|
||||
|
||||
**1-day manual-work floor, as requested:** any raw commit-clustered
|
||||
estimate below `MANUAL_EFFORT_FLOOR_DAYS = 1.0` is floored to 1.0 day
|
||||
(never reported smaller) and flagged with an explicit warning explaining
|
||||
this is very likely a measurement gap — commit-clustering is a floor
|
||||
estimate by design — and should usually be compensated for by a manual
|
||||
override rather than trusted at face value. A second, independent
|
||||
sanity-check warning fires when a repo's finished-workplan/task volume is
|
||||
substantial but the raw time estimate is still low, catching the case
|
||||
where the floor itself wasn't triggered but the estimate still looks
|
||||
implausible (demonstrated live: running the calculator against
|
||||
`target-revenue`'s own history — 7 finished workplans, 57 finished
|
||||
tasks — correctly flagged its 2.38-day raw estimate as under-counted,
|
||||
above the 1-day floor but still clearly too low for that much finished
|
||||
work).
|
||||
|
||||
`scripts/effort_calculator_cli.py` — a CLI wrapper printing the estimate
|
||||
as JSON, following the same offline-first, no-Phase-declaration pattern
|
||||
as `scripts/trf_onboard.py`. `tests/test_effort_calculator.py` (15 tests,
|
||||
deterministic — builds throwaway git repos and directory fixtures under
|
||||
`tmp_path` rather than depending on any real repo's changing state)
|
||||
covers commit clustering (empty/single-commit/multi-session), workplan/
|
||||
task parsing, size-metric exclusion, token-cost pricing, the floor-and-
|
||||
warning behavior, the independent sanity-check warning, and an
|
||||
end-to-end smoke test. Full suite: 79 passing offline (64 + 15 new), no
|
||||
new hard dependency (stdlib + existing `pathlib`/`subprocess`/`re` only).
|
||||
|
||||
## Apply calculator to real candidate repos
|
||||
|
||||
```task
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue