this post was submitted on 27 Jul 2024

222 points (96.6% liked)

Technology

59589 readers

2838 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

222

Deleted GitHub data is forever accessible to anyone, researchers claim | Cybernews (cybernews.com)

submitted 4 months ago by AnActOfCreation@programming.dev to c/technology@lemmy.world

16 comments fedilink hide all child comments

“The implication here is that any code committed to a public repository may be accessible forever as long as there is at least one fork of that repository,” the report’s authors claim.

Am I dumb or is this exactly the purpose of forks? I feel like I'm missing something.

all 17 comments

sorted by: hot top controversial new old

[–] Dave@lemmy.nz 59 points 4 months ago (1 children)

The article is really not clear. Is it saying if a project is forked, then the original is made private, the fork can access data from the private fork?

potentially enabling malicious actors to access sensitive information such as API keys and secrets even after users think they’ve deleted it.

Is this saying people misunderstand git and think committing a deletion makes people unable to access the previous version? Or is it saying the sharing between public and private repos can expose keys in private repos?

If you accidentally commit an API key into a public repository... you need to roll that key. Even if it was deleted completely, someone still could have accessed it while it was there.

[–] eager_eagle@lemmy.world 14 points 4 months ago (1 children)

from their actual report

As long as one fork exists, any commit to that repository network (ie: commits on the “upstream” repo or “downstream” forks) will exist forever.

   This further cements our view that the only way to securely remediate a leaked key on a public GitHub repository is through key rotation. We’ve spent a lot of time documenting how to rotate keys for the most popularly leaked secret types - check our work out here: howtorotate.com.

[–] Dave@lemmy.nz 11 points 4 months ago (2 children)

I'm still not sure that answers it. If I fork a project, and the upstream project commits an API key (after I've forked it), then they delete the commit, does this commit stay available to me (unexpected behaviour)? Or is it only if I sync that commit into my repo while it's in the upstream repo (expected behaviour)?

Or is it talking about this from a comment here:

Word of caution 2: The commit can still be accessible directly via SHA1. Force push does not delete the commit, it creates a new one and moves the file pointer to it. To truly delete a commit you must delete the whole repo.

Someone replies and said by having garbage collection kick in it removes this unconnected commit, but it's not clear to me whether this works for github or just the local git repo.

Perhaps the issue is that these commits are synced into upstream/downstream repos when synced when they should not be?

Like I said, I'm really confused about the specifics of this.

[–] Morphit@feddit.uk 7 points 3 months ago (1 children)

I think Github keeps all the commits of forks in a single pool. So if someone commits a secret to one fork, that commit could be looked up in any of them, even if the one that was committed to was private/is deleted/no references exist to the commit.

The big issue is discovery. If no-one has pulled the leaky commit onto a fork, then the only way to access it is to guess the commit hash. Github makes this easier for you:

What’s more, Ayrey explained, you don’t even need the full identifying hash to access the commit. “If you know the first four characters of the identifier, GitHub will almost auto-complete the rest of the identifier for you,” he said, noting that with just sixty-five thousand possible combinations for those characters, that’s a small enough number to test all the possibilities.

I think all GitHub should do is prune orphaned commits from the auto-suggestion list. If someone grabbed the complete commit ID then they probably grabbed the content already anyway.

[–] Dave@lemmy.nz 2 points 3 months ago (1 children)

Thanks, I think that explains it a bit more. It is unexpected to me, as a non-git expert, and I'm sure many others.

[–] Morphit@feddit.uk 2 points 3 months ago

I guess the funny thing is that each Git commit is internally just a file. Branches and tags are just links to specific commit files and of course commits link to their parents. If a branch gets deleted or jumped back to a previous commit, the orphaned commits are still left in the filesystem. Various Git actions can trigger a garbage collection, but unless you generate huge diffs, they usually stick around for a really long time. Determining if a commit is orphaned is work that Git usually doesn't bother doing. There's also a reflog that can let you recover lost commits if you make a mistake.

[–] grrgyle@slrpnk.net 5 points 3 months ago (1 children)

In my experience with GitHub, dropped commits remain indefinitely accessible. I use this to my advantage on pull requests with lots of good commit context that I don't want totally lost in a squash: by copying result of git log --oneline main... into the PR body. The SHAs remain accessible even after I force push my branch down to a single commit.

I think there is a theoretical limit to how long these commits remain accessible, but I haven't ever hit it in my daily usage.

[–] Dave@lemmy.nz 2 points 3 months ago

Ah thanks, this explains it a bit more.

[–] gencha@lemm.ee 13 points 3 months ago

What did they research? The git and GitHub documentation, and then the manual on clickbaiting? The shit people publish as research these days to boost their profile...

[–] hedgehog@ttrpg.network 13 points 4 months ago

The concern is that branches and commits that are not otherwise publicly visible become visible, thanks to the way Github handles forks.

[–] Boozilla@lemmy.world 7 points 4 months ago (2 children)

I find the language of the article a little confusing, too. But I'm a noob when it comes to GitHub. I have a couple of private projects that I've never shared, so not sure if this applies to me or not. We also plan on transitioning to GitHub at work. It's always smart to assume nothing ever gets truly deleted from an internet service.

[–] starshipwinepineapple@programming.dev 4 points 3 months ago* (last edited 3 months ago)

If you only ever keep your repository private AND it is not a fork of a public repo, then you are fine. Full stop.

If you ever fork the repo and make a "INTERNAL" private fork but move the main project public then anything you commit to the private fork will be discoverable through the public project.

Basically you should assume if you make a repo public then the repo and all of its forks will be public-- even if the forks are "private" the commit data can be found through the main repo.

[–] Aatube@kbin.melroy.org 7 points 4 months ago* (last edited 4 months ago)

*if it was forked or is a fork of something public. Only commits made while public can be read.

I feel like I’m missing something

You should read the original research article, which has its own thread someone else linked below. Basically, people often delete a fork after testing the public repo by committing an API key, which can be read using the method mentioned, which GitHub claimed was an intentional design feature.

[–] hedgehog@ttrpg.network 6 points 4 months ago

Here’s a link to an earlier discussion on this topic: https://lemmy.ml/post/18368342

[–] hexagonwin@lemmy.sdf.org 4 points 3 months ago

if it was never public/forked i guess it's alright..?

[–] starshipwinepineapple@programming.dev 4 points 3 months ago

The original report: https://trufflesecurity.com/blog/anyone-can-access-deleted-and-private-repo-data-github