How much the code in Bitcoin protocol changed in 2024?

[fillippone]

Hello,
I am absolutely not a technical person, but I stumbled in this slide from River:





276,000 changes in line of codes seems a very high number to me.

According to ChatGPT:

The Bitcoin protocol consists of several components, mainly the Bitcoin Core client, which is the reference implementation. As of recent updates, Bitcoin Core has around 600,000 to 700,000 lines of code, depending on the version.

This means that in 2024 almost 40% of the code changed.
This seems a lot to me, is this information reliable? How can I verify this?

[She shining]

This means that in 2024 almost 40% of the code changed.
This seems a lot to me, is this information reliable? How can I verify this?

You can verify this from their github repository https://github.com/bitcoin/bitcoin and clicking on insight(click on the... at the top corner if it's not visible) would grant access to various data.
What you need should be in code frequency but github limits visualization for massive projects like Bitcoin.

So you can check on contributors and it would show their commits( saved file of changes).
Specify the duration to let's say 12 months and you can see their commit and to check manually. Click on their profile and you would see some project they have worked on. Click on Bitcoin Core and select the Year. Yes quite time consuming but the essence is to show that it's indeed possible for such changes to have been made not to verify everything.

If it's still not clear. I could use image though.

[ABCbits]

That PDF screenshot have misleading information.
1. Not all lines of code on Bitcoin Core manage Bitcoin protocol, some of them have different usage (such as wallet management and how blockchain data is stored locally).
2. A Bitcoin Core developer state over 70% activity is related with maintenance[1].
3. With Ctrl-F, i unable to find mention whether "Over 276,000" include comment (rather than actual code) and refactor (rewrite the code without change on it's behavior).

And in narrow sense, Taproot was most recent major protocol change.

[1] https://www.coindesk.com/consensus-magazine/2023/02/23/can-bitcoin-afford-to-maintain-its-core-network-coindesk

[pooya87]

The protocol itself can only change through a fork (soft or hard fork) and we have only had about a dozen of them so far in the past 16 years, from BIP-16 (P2SH) all the way to BIP-341-342-343 (Taproot).

[MeGold666]

3. With Ctrl-F, i unable to find mention whether "Over 276,000" include comment (rather than actual code) and refactor (rewrite the code without change on it's behavior).

Since the Bitcoin protocol is written in stone, they at least have to pretend they're doing something.



They all have two things in common: they changed the world, and they are outdated.

[apogio]

Hello fillippone!

Out of curiousity, I asked git about these stats and I am posting the responses below.

Please take them with a grain of salt!

I am using the master branch for the statistics.


The first commit of 2024 was made on Jan 3rd, 2024 and the hash was 65c05db660b2ca1d0076b0d8573a6760b3228068:

Code:

> git log 65c05db660b2ca1d0076b0d8573a6760b3228068

commit 65c05db660b2ca1d0076b0d8573a6760b3228068
Merge: c3038bf95a b1318dcc56
Author: Gloria Zhao <[email protected]>
Date:   Wed Jan 3 11:16:48 2024 +0000

    Merge bitcoin/bitcoin#29013: test: doc: follow-up #28368

    b1318dcc56a0181783ee7ddbd388ae878a0efc52 test: change `m_submitted_in_package` input to fuzz data provider boolean (ismaelsadeeq)
    5615e16b705d74bf6ebb7c39523844f97a41cb6f tx fees: update `m_from_disconnected_block` to `m_mempool_limit_bypassed` (ismaelsadeeq)
    fcd429664818f14cace580513e7e6159335b5416 doc: fix typo and update incorrect comment (ismaelsadeeq)
    562664d26374331d291b97e2e2f7fca1f0fd467b test: wait for fee estimator to catch up before estimating fees (ismaelsadeeq)

    Pull request description:

      This is a simple PR that does two things
      1.   Fixes #29000 by waiting for the fee estimator to catch up after `removeForBlock` calls before calling `estimateFee` in the `BlockPolicyEstimates` unit test.

      2. Addressed some outstanding review comments from #28368
      - Updated `NewMempoolTransactionInfo::m_from_disconnected_block` to `NewMempoolTransactionInfo::m_mempool_limit_bypassed` which now correctly indicates what the boolean does.
      - Changed  input of `processTransaction`'s tx_info  `m_submitted_in_package` input from false to fuzz data provider boolean.
      - Fixed some typos, and update incorrect comment

    ACKs for top commit:
      martinus:
        re-ACK b1318dcc56a0181783ee7ddbd388ae878a0efc52
      glozow:
        utACK b1318dcc56a0181783ee7ddbd388ae878a0efc52

    Tree-SHA512: 45268729bc044da4748fe004524e0df696d2ec92c5bd053db9aad6e15675f3838429b2a7b9061a6b694be4dc319d1782a876b44df506ddd439d62ad07252d0e1

The last commit of 2024 was made on December 31, 2024 and the hash was 2bdaf52ed1259fd3bec22b680e12563fcee0a8b3:

Code:

> git log 2bdaf52ed1259fd3bec22b680e12563fcee0a8b3

commit 2bdaf52ed1259fd3bec22b680e12563fcee0a8b3
Author: Hennadii Stepanov <[email protected]>
Date:   Tue Dec 31 10:14:02 2024 +0000

    doc: Update NetBSD Build Guide

    1. Update the documented NetBSD version.

    2. Add the optional ZeroMQ package to align the guide with other *BSD
    systems.

    3. Update the Python version to meet the minimum requirement specified
    in https://github.com/bitcoin/bitcoin/pull/30527.

    4. Install `net/py-zmq` package to enable the `interface_zmq.py`
    functional test.

    5. Fix a formatting issue.

Between these commits, there have been many changes, you can get an idea about how many using the following command:

Code:

> git diff --shortstat 65c05db660b2ca1d0076b0d8573a6760b3228068 2bdaf52ed1259fd3bec22b680e12563fcee0a8b3

 1711 files changed, 130696 insertions(+), 79353 deletions(-)

The total number of lines of code that bitcoin has is 921267.

Code:

> git ls-files | xargs wc -l

921267 total

The total number of files that bitcoin has is 2768.

Code:

❯ git ls-files | wc -l

2768

[fillippone]

<...>
Between these commits, there have been many changes, you can get an idea about how many using the following command:

Code:

> git diff --shortstat 65c05db660b2ca1d0076b0d8573a6760b3228068 2bdaf52ed1259fd3bec22b680e12563fcee0a8b3

 1711 files changed, 130696 insertions(+), 79353 deletions(-)

The total number of lines of code that bitcoin has is 921267.

Code:

> git ls-files | xargs wc -l

921267 total

According to your statistics, there were 130K insertions and 80K deletions, so the claim of "276,000 lines of code changes" seems a lot.
Even considering refactoring or multiple changes over the same lines of code, the total seems too big compared to the total number of lines of code: 921267.
Seems strange, as River is a pretty reputable source of information.

[apogio]

According to your statistics, there were 130K insertions and 80K deletions, so the claim of "276,000 lines of code changes" seems a lot.
Even considering refactoring or multiple changes over the same lines of code, the total seems too big compared to the total number of lines of code: 921267.
Seems strange, as river is quite a reputable source of information.

River is reputable indeed, although I don't have personal experience with them.

Well, as you said refactoring is something that can't be easily measured because the 130K insertions and 80K deletions are aggregated values.
You can't be perfectly sure about the number of lines that have been added or removed.
It would be better if they had given us the branches on which they have made their calculations. I reckon they could have taken all the Pull Requests of 2024 and have counted the number of lines, for the PRs that have been merged on master.
I am definitely curious to know how they came up with this number. If you have any info on this, let me know. Because, indeed, 276,000 lines are too many. Which doesn't mean that it's not true, but I am just curious.

[apogio]

Sorry for double posting, I normally don't do it, but I want it to be clearly seen here 

EDIT:

I 've found the way they calculated it.

According to Jameson Lopp^[1], the command they used was:

Code:

 git log --no-merges --after="2023-12-31" --before="2024-12-31" --stat | grep -Eo '[0-9]+ insertions|[0-9]+ deletions' | awk '{sum+=$1} END {print "Total lines changed:", sum}'

which, I just ran and gave me this response:

Code:

Total lines changed: 276826

Essentially what it does is:

1. it takes all the commits after 2023-12-31 and before 2024-12-31 (i.e. in 2024), with this git log --no-merges --after="2023-12-31" --before="2024-12-31" --stat
2. the result for each log will have the commit message, the commit author, the description and details in this format: 2 files changed, 2 insertions(+), 2 deletions(-)
3. it uses grep -Eo '[0-9]+ insertions|[0-9]+ deletions' to get the details in this format: 45 deletions, 91 insertions etc.
4. it uses awk '{sum+=$1} END {print "Total lines changed:", sum}' to aggregate all the numbers. For example: 45 + 91 = 136 lines.

I don't like this command, but it's Jameson Lopp who did it, so I can't say a lot. He 's more trustworthy than I am.

[1] https://blog.lopp.net/bitcoin-2024-annual-review/


Small silly knowledge of the day! I hope you enjoyed it as much as I did

[pooya87]

It might be worth knowing that Git is not exactly reporting amount of code changes and the X number of insertions and Y number of deletions that is reported is not necessarily total number changes in "lines of code". Take this commit for instance for better understanding of what I mean:
https://github.com/bitcoin/bitcoin/commit/c7d5dcaa61471cc26c3fb1772dfc4c82b20e1f9d#diff-c42097cc877be0016cc04ac42254466b267533286801646a2100db673b806e67
It is a typo fix commit and this particular part of it is simply adding the letter "t" to a comment (ie. commiting to committing) inside src/txgraph.h file, but git counts that as 2 changes: 1 deletion and 1 addition (1 line deleted, 1 line added).

[apogio]

It might be worth knowing that Git is not exactly reporting amount of code changes and the X number of insertions and Y number of deletions that is reported is not necessarily total number changes in "lines of code". Take this commit for instance for better understanding of what I mean:
https://github.com/bitcoin/bitcoin/commit/c7d5dcaa61471cc26c3fb1772dfc4c82b20e1f9d#diff-c42097cc877be0016cc04ac42254466b267533286801646a2100db673b806e67
It is a typo fix commit and this particular part of it is simply adding the letter "t" to a comment (ie. commiting to committing) inside src/txgraph.h file, but git counts that as 2 changes: 1 deletion and 1 addition (1 line deleted, 1 line added).

Sure, it's not a good (accurate) metric. You need to take this into account.
I don't think Git can provide a more accurate count however, because technically it's a pointer-based system. So, the change in the commit you posted is a modification of the same line but for Git something was deleted and something else was added.