Compare two projects?

[kaihendry]

Use case is that given two projects, which can be objectively be said to be "lighter weight"?

gh repo clone openai/codex
gh repo clone badlogic/pi-mono

This output is way too noisy:

lightweight $ sloc pi-mono/packages/coding-agent/
───────────────────────────────────────────────────────────────────────────────
Language            Files       Lines    Blanks  Comments       Code Complexity
───────────────────────────────────────────────────────────────────────────────
TypeScript            317      80,008     9,951     7,149     62,908      9,296
Markdown               57      14,732     3,781         0     10,951          0
JSON                   13         833        12         0        821          0
JavaScript              4       2,930       212       203      2,515        339
JSONL                   2       2,022         0         0      2,022          0
Shell                   2         245        31        20        194         18
C                       1          72        12        12         48          4
CSS                     1       1,001       163        23        815          0
HTML                    1          55         4         3         48          0
───────────────────────────────────────────────────────────────────────────────
Total                 398     101,898    14,166     7,410     80,322      9,657
───────────────────────────────────────────────────────────────────────────────
Estimated Cost to Develop (organic) $2,701,914
Estimated Schedule Effort (organic) 20.07 months
Estimated People Required (organic) 11.96
───────────────────────────────────────────────────────────────────────────────
Processed 6922718 bytes, 6.923 megabytes (SI)
───────────────────────────────────────────────────────────────────────────────
lightweight $ scc codex/
───────────────────────────────────────────────────────────────────────────────
Language            Files       Lines    Blanks  Comments       Code Complexity
───────────────────────────────────────────────────────────────────────────────
Rust                1,435     634,264    53,662    20,476    560,126     30,491
TypeScript            457       7,421     1,204     2,265      3,952        228
Plain Text            443       7,271       699         0      6,572          0
JSON                  228      89,016         3         0     89,013          0
Markdown              154      14,783     3,575         0     11,208          0
TOML                  111       4,137       450       258      3,429         19
Bazel                 102       1,887       223        51      1,613         35
Python                 82      22,017     3,568       418     18,031      2,180
YAML                   40       4,751       492       203      4,056          0
Patch                  29       2,702       146         0      2,556          0
Shell                  29       3,151       428       280      2,443        357
SQL                    26         273        21         0        252          0
Meson                   7         385        36         6        343         30
C                       6       5,945       955       645      4,345      1,525
JavaScript              6       2,169       207       112      1,850        319
C Header                5         297        39        56        202          7
SVG                     5          17         0         0         17          0
License                 4         718       112         0        606          0
Powershell              3         699       118        70        511        124
XML                     3         663        13         0        650          0
Dockerfile              2          86        16        13         57         13
HTML                    2         320         5         0        315          0
Nix                     2         143         8        10        125          3
Smarty Template         2         112        16         0         96         35
Autoconf                1           1         0         1          0          0
Docker ignore           1           1         0         0          1          0
Emacs Lisp              1           1         0         0          1          0
JSONL                   1       8,041         0         0      8,041          0
Jupyter                 1         587         0         0        587          0
───────────────────────────────────────────────────────────────────────────────
Total               3,188     811,858    65,996    24,864    720,998     35,366
───────────────────────────────────────────────────────────────────────────────
Estimated Cost to Develop (organic) $27,066,125
Estimated Schedule Effort (organic) 48.17 months
Estimated People Required (organic) 49.92
───────────────────────────────────────────────────────────────────────────────
Processed 28626595 bytes, 28.627 megabytes (SI)
───────────────────────────────────────────────────────────────────────────────

I also feel the cost to develop metric in light of the AI agent stuff makes no sense. --no-cocomo should be the default. Listing the languages also arguably doesn't matter.

[boyter]

Yeah the ability to compare is something I have wanted to add. I might look into doing it.

As for the defaults... You could try the new locomo ;)

I am thinking of adding a dotfile support so you can set it up how you like for this.

[kaihendry]

Interesting!! You need to release it? Tbh I'd release this under some experimental flag since this surely depends on the nature of the project, these variables will vary wildly. e.g. model cost and review time.

I wonder how you could go about simply collecting research information with the same rigours of COCOMO. Though a part of me has never really given it much credence, I much prefer the simpler LOC metric. This is why I penned https://github.com/kaihendry/lesshero with no thought to COCOMO.

[boyter]

Yes, I had planned on holding the release for a bit till I get some other things I want in. I did write about it here though https://boyter.org/posts/sloc-cloc-code-locomo-llm-output-cost-model/

Honestly... its a test. I threw it out and hopefully someone comes and improves it. At least something has it at this point.

[boyter]

So... are you looking for something like cloc?

$ cloc --diff processor/ scripts
      28 text files.
       2 text files.
       1 file ignored.

github.com/AlDanial/cloc v 2.06  T=0.08 s (367.2 files/s, 313772.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Go
 same                            0              0              0              0
 modified                        0              0              0              0
 added                           1             22              5             88
 removed                        28           1627           1005          22031
-------------------------------------------------------------------------------
SUM:
 same                            0              0              0              0
 modified                        0              0              0              0
 added                           1             22              5             88
 removed                        28           1627           1005          22031
-------------------------------------------------------------------------------

[boyter]

OK I had a look through what cloc codes and got this,

Project Comparison: cloc approach vs proposed scc approach

  cloc's approach

  cloc uses Algorithm::Diff::sdiff() to perform line-level diffing on matched file pairs. For each pair of files that share the same relative path, it:

  1. Strips blanks and comments from both files to classify each line as code/comment/blank
  2. Runs sdiff() on the code lines (L vs R) to tag each line as same/added/removed/modified
  3. Runs sdiff() on the comment lines (L vs R) with the same tagging
  4. Computes blank line differences separately

  This produces four categories with absolute counts:

  - same: lines identical in both files (and files only present in both sets with identical content)
  - modified: lines that exist in both files at the same position but with different content (requires sequence alignment to determine)
  - added: lines only in set 2, plus all lines from files only in set 2
  - removed: lines only in set 1, plus all lines from files only in set 1

  The same/modified rows are symmetric (same result regardless of argument order). The added/removed rows swap when arguments are reversed. All values are absolute counts.

  cloc requires a --diff-timeout (default 10 seconds per file) because sdiff() can be extremely slow on large files with many repeated lines. It runs 3 separate sdiff() passes per modified file pair.

  Proposed scc approach

  scc already classifies every line as code/comment/blank during its normal counting pass, so the per-line classification step that cloc needs to do is already done. Rather than performing line-level sequence alignment, scc would:

  1. Run the normal counting pipeline on both sets
  2. Match files by relative path
  3. Files only in set 1 -> "removed" (all their counts)
  4. Files only in set 2 -> "added" (all their counts)
  5. Files in both sets -> "common" (show the delta between the two counts)

  This produces three categories:

  - added: absolute counts from files only in set 2
  - removed: absolute counts from files only in set 1
  - common: net delta (positive or negative) of counts between matched files, showing the number of common files rather than a file count delta

  The common row uses signed values (+/-) to indicate direction of change, where the first argument is the baseline. scc --diff old/ new/ with +187 code means 187 more code lines in new/ than old/.

  Key differences

  ┌───────────────────────────────┬──────────────────────────────────────────────────┬───────────────────────────────────────────────┐
  │                               │                       cloc                       │                scc (proposed)                 │
  ├───────────────────────────────┼──────────────────────────────────────────────────┼───────────────────────────────────────────────┤
  │ Line-level diffing            │ Yes, via Algorithm::Diff::sdiff()                │ No                                            │
  ├───────────────────────────────┼──────────────────────────────────────────────────┼───────────────────────────────────────────────┤
  │ Categories                    │ same, modified, added, removed                   │ added, removed, common                        │
  ├───────────────────────────────┼──────────────────────────────────────────────────┼───────────────────────────────────────────────┤
  │ "Modified" detection          │ Can identify individual changed lines            │ Cannot distinguish; shows net delta instead   │
  ├───────────────────────────────┼──────────────────────────────────────────────────┼───────────────────────────────────────────────┤
  │ Values in common/matched rows │ Absolute line counts per category                │ Signed deltas (+/-)                           │
  ├───────────────────────────────┼──────────────────────────────────────────────────┼───────────────────────────────────────────────┤
  │ Complexity metric             │ Not available                                    │ Included                                      │
  ├───────────────────────────────┼──────────────────────────────────────────────────┼───────────────────────────────────────────────┤
  │ Performance risk              │ sdiff() can be very slow; needs per-file timeout │ No diffing overhead; just two counting passes │
  ├───────────────────────────────┼──────────────────────────────────────────────────┼───────────────────────────────────────────────┤
  │ Passes over matched files     │ 3x sdiff() per file pair                         │ Simple count subtraction                      │
  └───────────────────────────────┴──────────────────────────────────────────────────┴───────────────────────────────────────────────┘

  What scc gains

  - Complexity delta (cloc cannot provide this)
  - No pathological performance cases from diff algorithm
  - Simpler implementation: two normal scc runs + file path matching + arithmetic

  What scc loses

  - Cannot report "same" vs "modified" line counts for matched files (only the net delta)
  - Cannot say "3 lines were modified" — only "net +20 code lines across common files"

scc is NOT a diff tool, hence me thinking to not do what cloc does when it comes to diff.

Rather than diff it is telling you the difference in size between two projects, with the first argument being the one we are measuring against.

cloc diffs the content of files, scc compares the size of projects

[kaihendry]

Hey Ben, I ended up "vibing" what I wanted for the most part: https://github.com/kaihendry/cream

$ ./cream ~/tmp/lightweight/codex/ ~/tmp/lightweight/pi-mono/
Lightest projects first
Project                                 Files  >1000    SLOC  MaxFile  Bar
-------------------------------------------------------------------------------------------------
/Users/hendry/tmp/lightweight/pi-mono/    796     32  216504    14277  ######
/Users/hendry/tmp/lightweight/codex/     3647    177  846089    15587  ########################

Cream rises to the top: /Users/hendry/tmp/lightweight/pi-mono/ has 629585 fewer SLOC than /Users/hendry/tmp/lightweight/codex/, 145 fewer files over 1000 lines, and a largest file that is 1310 lines smaller.

I don't care about the language. I also got the idea to point out files >1000 SLOC since that's considered bad practice for LLVMs too.