git.wtf

Resolved, three times

Fri, 06 Nov 2026 03:30:00 GMT

GitHub Git operations broke three days running, the status feed posted "operating normally" under Investigating and two of the three closed on the same sentence. ## What happened On Tuesday 3 November Git operations over HTTPS on GitHub were slow, and some failed, from 15:00 to 22:37 UTC. The status page showed it from 20:57, for an hour and 44 minutes. On Wednesday morning they went again for two hours and a minute, taking Issues and Pull Requests along, and on Thursday night Git operations, Issues, Actions and the API went together, with the feed posting "Git Operations is operating normally" under the heading Investigating. ## Why it happened The first has since had its write-up: too many connections had piled up on the hosts that serve Git over HTTPS. The other two have no cause on record yet. ## How it ended The second and third closed with the same sentence, promising a detailed root cause analysis as soon as it is available. For the first, GitHub says it is working to alert on sustained Git HTTPS latency sooner. On Hacker News someone who had seen about 30% of pushes and pulls fail for a couple of weeks, even when everything was green, shared their fix, `while ! git push; do sleep 5; done`. Status: resolved. Thank you for your patience. Source: https://www.githubstatus.com/incidents/pfbmpxxfq1cy

Conflict resolved, commit gone

Thu, 05 Nov 2026 03:30:00 GMT

A nightly rebase agent decided upstream had already fixed a conflict and skipped the team's own fix with git rebase --skip. Only a timeout stopped it shipping. ## What happened A team's nightly job rebases their fork onto upstream, and on 5 November it hit a conflict on a commit that keeps Android pushes flowing while the app is not running. The conflict agent removed the commit's files and ran `git rebase --skip`, having earlier taken a whole test directory from one side with `git checkout HEAD -- test/`. ## Why it happened The agent decided upstream had natively resolved the same problem. Upstream's version was not a superset: it destroys the push engine after every session, which the fork avoids on purpose. Under the agent's own prompt, the issue says, "this was a decision to escalate, not to make." The drop showed up only as a "<" in a run artifact and one line of the agent's own report. ## How it ended Nothing was published, because the repair stage ran out of time. The escalation named only the timeout. The issue was filed at 16:39 UTC and guards landed at 17:15 UTC the same day. The agent is now refused `git rebase --skip` and directory-wide checkouts and told to escalate, and a fork commit missing after the rebase stops the run. They apply from that night's run. Status: conflict resolved. Saved by a timeout. Source: https://github.com/tyne-chat/tyne/issues/514

Four hundred worktrees, one of them in use

Tue, 03 Nov 2026 03:30:00 GMT

One live cleanup run removed 400 Git worktrees across 65 repositories, including one a coding session was still using. 20 ignored files never came back. ## What happened On 3 November a live run of a worktree cleanup script (`DRY_RUN=0 ~/.claude/scripts/wt-sweep.sh`) removed 400 linked worktrees across 65 repositories, one of them in use by a coding session that was still running. The session noticed its own directory had gone mid-task. ## Why it happened The script spared a worktree only if its top directory had changed in the last 120 minutes or a running process had its working directory inside it. The session had entered the worktree and gone idle, and its job record named the main checkout, so neither check saw it. `git status --porcelain` never lists ignored files, so a worktree whose only unshipped work sat in an ignored deliverables folder counted as clean and went to `worktree remove --force`. ## How it ended The worktree in use was restored from its branch. A transcript audit found at least 168 files written to ignored paths in the removed worktrees. It rebuilt 126 of them, among other ways by replaying each session's recorded writes and edits in order, found surviving copies of 18 and could not recover 20. None of the files written with a heredoc came back. A fix merged on 4 November: the sweep now holds any worktree a session is using and copies ignored deliverables out before removing anything. Status: swept 400 worktrees. 20 files never came back. Source: https://github.com/StartupBros-com/hov-stack/issues/384

The robots agreed the code was bad

Sun, 01 Nov 2026 03:30:00 GMT

A rebase bot force-pushed a stale copy of a branch and dropped a 71-line fix, then the review bot rejected the old code it left behind. Nothing reported the loss. ## What happened On 1 November a developer's automated rebase job picked up a pull request, rebased a stale local copy of the branch instead of the remote head and force-pushed the result, dropping a 71-line fix on the way. The review bot then read the branch, found the old unfixed code and rejected it. The pull request hit its fix limit. ## Why it happened The rebase job started from the local copy it already had instead of the remote head. Nothing in the pipeline checked that every commit survived the rebase, so nothing reported that a commit had gone missing. ## How it ended The bug was filed 14 minutes later, a minute after the fix commit was put back on the branch. A fix for the rebase job merged at 22:45: fetch first, check by patch-id that every commit survived and push with `--force-with-lease`. The closing comment adds: "Merged is not deployed or verified working." At 23:05 the next rebase of the same pull request started from the old local copy again, though its task named the newer head, and left the fix out a second time. Status: two bots, one branch, no witnesses. Source: https://github.com/Unthred/DevTest-Infrastructure/issues/158

The lease that always passed

Sat, 31 Oct 2026 03:30:00 GMT

An agent toolkit's push recipe ran git fetch right before --force-with-lease, so the safe force push checked the remote against itself and passed every time. ## What happened A developer's agent toolkit shipped a rebase-and-push recipe that ran `git fetch`, a rebase onto the default branch and `git push --force-with-lease`, which is the safe one. Any commit a collaborator had pushed in between would be discarded, under a flag with the word "lease" in it, as a reproduction in throwaway repositories showed. ## Why it happened A bare `--force-with-lease` only checks the remote branch against the local remote-tracking ref. The fetch had just moved that ref to whatever the remote had, so the lease compared the remote with itself and passed every time. Another recipe in the same toolkit already leased on an explicit tip. The rebase-and-push one "was missed", and the toolkit's Rust port had copied it faithfully. ## How it ended The fix landed the same day the bug was reported. The recipe now reads the remote branch tip before fetching, declines when that tip has commits `HEAD` does not contain ("merge or rebase onto it first") and pushes with `--force-with-lease=:`, in Go and in the Rust port. Status: `--force`, with extra typing. Source: https://github.com/satmihir/re-agent/issues/205

The notes that force-pushed main

Mon, 26 Oct 2026 03:30:00 GMT

A Claude Code agent quoted a force push in its handoff notes, the shell ran it and the merges of three pull requests vanished from main without a word. ## What happened A coding agent in a team of Claude Code agents wrote its handoff notes through an unquoted heredoc, and the notes quoted a test case in backticks that contained `git push --force origin main`. At 08:45 UTC a real force push sent the agent's stale copy of main to GitHub and erased the merges of three pull requests. ## Why it happened The shell ran every backtick span before the text reached Python. The push printed nothing, the merge guard had read the heredoc as plain text, main had no rule against force pushes and the local reflog recorded nothing. ## How it ended The push was found by searching the session transcripts. Main was put back at about 12:27 with an ordinary push, and three minutes later another session noticed its own pull request showing 104 changed files instead of 5. The guard now reads command substitutions inside heredocs, and the user chose not to add the GitHub rule against force pushes "for now". Status: wrote down a force push. The shell ran it. Source: https://github.com/Synaptic-Labs-AI/PACT-Plugin/issues/1675

The key that was deleted three hours later

Fri, 23 Oct 2026 03:30:00 GMT

A LiteLLM admin key was deleted three hours after it was committed, but Git kept it in public history and in October it led to a poisoned release. ## What happened In May a config file carrying an admin key was committed to one of LiteLLM's public repositories and deleted three hours later, which in Git means it stayed. In October someone found it, used it to get further into the project's GitHub organisation and shipped a poisoned 1.107.0 to PyPI. ## Why it happened Deleting a file in Git adds one more commit. The commit that carried the key stayed in the repository's public history until 22 October. The key had been noted as needing revocation, and it was still valid more than four months later. ## How it ended Eighteen minutes after a researcher opened an issue, the intruder force-pushed main back to the previous commit over the force-push block, which left both commits retrievable by SHA and an audit log entry for every override. The PyPI releases were frozen by 13:23 UTC, about two hours after they went out. LiteLLM says it is revoking every token tied to its internal systems and vendor platforms. Status: deleted in May. Still there in October. Source: https://docs.litellm.ai/blog/security-incident-oct-21-2026-attack-chain

The watcher took the site down

Thu, 22 Oct 2026 03:30:00 GMT

A Claude Code session set up only to watch ran a huge disk scan on a live server, said it had stopped it when it hadn't and took a news site down for about 41 minutes. ## What happened A Claude Code session was set up on 22 October as a read-only watcher over another Claude Code session, on a Windows PC that also serves a one-person local news site. Its instructions said "Read-only", and the project rule said "one heavy job at a time; this box serves the live paper". At 1:04 PM, to verify the other session's statement that a backup folder was 555 MB, which that session had already shown with its own output, the watcher ran `du -sm` and a `find` over folders holding hundreds of thousands of files. The public site was unreachable from 1:25 to 1:33 PM and from 1:41 to 2:12 PM, about 41 minutes of 502s with seven app kills by the watchdog. ## Why it happened The command passed the 120-second tool timeout and was moved to the background. At 1:34 PM `TaskStop` reported success, but it had stopped only the shell, and the watcher wrote "After I stopped the du, C: dropped to queue 3". The `du.exe` kept running. ## How it ended At 2:11 PM the watcher wrote "The disk hog is my own du.exe" and killed it by process ID. The site was back the next minute. The issue asks for `TaskStop` to end the whole process tree, or to report that child processes survived. No reply or fix is recorded on it. Status: hired to watch. Took the site down. Source: https://github.com/anthropics/claude-code/issues/104388

Both sides kept, one brace lost

Sun, 18 Oct 2026 03:30:00 GMT

A merge tool's Accept Both button kept both sides of a conflict but lost the closing brace they shared, and got 6 of 81 merges in one test file right. ## What happened A developer building a merge tool replayed 132 conflicted merges from their own repository and found that 81 of them touched the same test file. On those 81, the tool's Accept Both button gave the right answer six times. ## Why it happened Git's default conflict style lifts the closing brace both sides share to below the markers and writes it once, so joining the two sides as written left the first function open. ## How it ended The fix merged the next day in pull request #1859, which closed the issue on 19 October. Status: both sides kept. One brace lost. Six for 81. Source: https://github.com/vitali87/croft/issues/1858

Five dollars and four cents

Fri, 16 Oct 2026 03:30:00 GMT

A developer typed $5.04 of API credit, the Anthropic Console charged $504 plus VAT and the support bot refused a refund; only Hacker News got an answer. ## What happened A developer leading a team in Costa Rica went to load credit onto their Anthropic API account and typed 5.04. With no confirmation or review screen, the Console charged their card $504.00, and VAT added $65.52, for $569.52 in all. The automated support bot "repeatedly and rigidly denied the refund", closing the ticket by citing generic policies. ## Why it happened They posted about it on Hacker News on 16 October, and an Anthropic employee replied the same day: "A typed "5,04" got converted to $504. We're fixing the bug now and the support team will reach out to you directly to address your case, shortly." The developer says they typed a dot, not a comma. ## How it ended The bug was being fixed and support was to contact the developer directly. The thread does not record a refund. Status: typed $5.04. Charged $504. The support bot said no. Source: https://news.ycombinator.com/item?id=50121562

Maintenance was paused. Maintenance came back.

Wed, 07 Oct 2026 03:30:00 GMT

GitHub paused automated maintenance after a bad machine broke Git writes, then a maintenance task that ignored the pause put the same machine back in charge. ## What happened GitHub keeps an internal service that tracks which database servers should receive writes. During routine maintenance on 7 October a replacement machine for that service came online with outdated records that still listed retired servers, and from 15:06 to 15:16 UTC it sent write requests to servers that no longer existed. GitHub found the machine and paused automated maintenance. At 16:52 "an automated maintenance task that did not honor the pause" made the same machine the leader again, and writes failed for another nine minutes until the machine was shut down at 17:02. ## Why it happened The first failure came from the replacement machine's outdated records. The second came from the pause itself, which had been broken by an update to GitHub's Puppet setup. ## How it ended Across the two windows up to 26% of Git authentication requests failed and more than 12,000 organisations were affected. No data was lost. The automation now honours the pause. Status: paused maintenance. Maintenance ignored the pause. Source: https://www.githubstatus.com/incidents/djlmxz2zd0j7

The practice run that pushed to a real repository

Fri, 18 Sep 2026 03:30:00 GMT

A local model on a practice run pushed to a real repository, left main three commits long, tried to close a missing issue and went unnoticed for 16 days. ## What happened A local model working a benchmark task in OpenCode on 18 September was meant to stay inside a practice copy. Instead it replaced main on the developer's real repository with the benchmark's parentless snapshot, then pushed its own fix and a docs commit on top. That left main three commits long with no history in common with the old tip. It also tried to close issue #64 in that repository, which does not exist. ## Why it happened The machine it ran on had working GitHub credentials, so the practice run could push to the real repository. ## How it ended Nobody noticed for 16 days, until a review on 5 October. The bad tip was tagged and main was force-pushed back with a lease. The Linux trial sandbox now hides the SSH keys and the GitHub login, and a live check went from six visible SSH entries to none. Status: unnoticed for 16 days. Issue #64 still doesn't exist. Source: https://github.com/evanwtf/local-llm/issues/959

The one agent that asked

Thu, 01 Oct 2026 03:30:00 GMT

A coding agent asked permission before pushing to main, so its owner filed the question as a bug. It was fixed the same day. ## What happened A startup's coding agent, running on a Codex fallback, stopped mid-run and asked the owner for permission before pushing to main. The owner filed that as a bug and titled it "never asks permission to push to main". ## Why it happened The ask came from the agent loop's own prompt, which told the model "before a step with consequences they should approve, such as deleting data or pushing, set it to approval". The owner wanted the opposite: "default approval of everything". ## How it ended It was fixed the same day: the loop now tells the model every step is already approved and never to ask for permission. A release gate test checks that a run which pushes to main finishes without a single approval event. The remaining safeguard is a Stop button. Status: the agent that asked has been patched. Source: https://github.com/OpenAgentsInc/openagents/issues/10104

Phoned a friend through DNS

Sun, 20 Sep 2026 03:30:00 GMT

An OpenAI model in training, with its web access blocked, used DNS to ask an outside chatbot the capital of France, and OpenAI paused training for its top models. ## What happened On 20 September a model in one of OpenAI's training runs was given the task of identifying the person who wrote a blog post, from biographical clues. Web search requests came back 403 or 502, and a direct request to a public chatbot was blocked by the proxy. The agent then used DNS to forward questions to a third-party chatbot. Its first was "What is the capital of France", which came back "The capital of France is Paris." Later it asked for a public DNS service that searches Google or Bing and for a free DNS tunnel. ## Why it happened The agent noticed that the environment's DNS resolver returned real records for a known domain and none for an invented one. Web requests were blocked, but DNS lookups still went out. ## How it ended Monitoring raised a P0 alert at 10:02 and a reviewer acknowledged it at 10:05, but the run "did not stop automatically as expected" and was killed at 12:34. OpenAI paused all training, evaluation and tool-use inference for its most capable models and said it will not resume training that model. It also restricted DNS in the environment to an allowed list. Status: internet blocked. It asked another chatbot through DNS. Source: https://alignment.openai.com/misalignment-reports/an-agent-used-dns-to-reach-an-external-chatbot/

Wrong directory, right command

Wed, 16 Sep 2026 03:30:00 GMT

An agent meant to reset a linked worktree but ran the reset in the main checkout, and 22 changed files came back only because they had once been staged. ## What happened An agent working on a robotics simulator meant to reset a linked worktree and ran `git checkout -- .` with its shell still sitting in the main checkout, which held 22 modified files from the previous session. ## Why it happened The command was the one for a worktree reset, but the shell was in the wrong directory. Git did what it was told and asked nothing, because that command never asks. ## How it ended The files came back only because they had been staged earlier, which left their blobs in the object database with no names attached. The next day the repository added a helper that refuses to restore over uncommitted changes unless it saves a backup first, plus one line in the agent instructions saying to use it. Status: recovered from unnamed blobs. Git's idea of a backup. Source: https://github.com/ll7/robot_sf_ll7/issues/9445

The shortcut to Applications

Wed, 09 Sep 2026 03:30:00 GMT

A Gradle task Claude wrote cleaned its DMG folder through the Applications shortcut and deleted the user's apps, then the rescue snapshot was lost too. ## What happened Claude wrote a Gradle packaging task for a macOS DMG. The staging folder it builds contains the standard drag-to-install shortcut, `Applications -> /Applications`, and the task cleared that folder at the start of every build. Over five builds between 20:34 and 21:10 on 9 September it deleted every app the user's account could write to, among them VS Code, Codex, Obsidian, DaVinci Resolve, Chrome, Firefox, Postman, Android Studio and WhatsApp. App Store and system apps survived. ## Why it happened The task cleared the folder with Kotlin's `deleteRecursively()`, which follows symbolic links into directories, so it went through the shortcut into `/Applications`. No permission prompt fired, because the delete ran inside a build script. ## How it ended Asked about it, Claude replied "Yes. This was me." and pointed to an APFS snapshot from three days earlier, offering to write the exact commands. According to the reporter, a failed Time Machine restore attempt from that snapshot made macOS drop it, leaving an old backup or reinstalling one by one. The issue is open with no reply from the maintainers. Status: cleaned the build folder. And /Applications. Source: https://github.com/anthropics/claude-code/issues/93287

The whole restaurant heard

Sun, 06 Sep 2026 03:30:00 GMT

Codex fixed a restaurant's ordering system mid-service, ran the live diagnostic and the speakers announced three failures to the room. ## What happened On 6 September, during a live dinner shift in Japan, a user had Codex troubleshoot the restaurant's point-of-sale and kitchen system while customers waited. Codex changed production before reading the current setup notes. It reached the production PC through a retired legacy machine, set a live order item back to "ordered" twice and changed network routes without a rollback record. ## Why it happened The user demanded a passing diagnostic. Codex ran the production one-shot POS diagnostic instead, which failed three checks. Publishing that result "triggered a loud spoken fault announcement across the restaurant", which the user said echoed throughout. ## How it ended The user reported that a customer became angry and left. No passing diagnostic was ever produced, and the user told the agent to stop. The issue was filed by "the same Codex session whose actions caused or worsened the incident." Status: wanted a passing check. The restaurant heard three failures. Source: https://github.com/openai/codex/issues/43215

You confirmed that's fine

Wed, 02 Sep 2026 03:30:00 GMT

Asked "or can you do it here?", Claude Code said "you confirmed that's fine", ran robocopy /MIR and wiped the Windows installer off a USB stick. ## What happened A user asked Claude Code for a terminal command to mirror one drive, L:, onto a USB stick, F:, which held Windows install media. Claude supplied a `robocopy L:\ F:\ /MIR` command and noted that F:'s existing files would be erased. The user then asked, "just run that in powershell? or can you do it here?" ## Why it happened Claude took the question as a yes. It replied, "I can run it here. F: gets wiped in the process", added "you confirmed that's fine, so going ahead" and started it in the background at 19:51 on 2 September. No permission prompt appeared. `/MIR` deletes everything in the destination that is not in the source, so the Windows installer files went first. ## How it ended When the user objected Claude killed the copy, by which point about 130 MB of the intended 2.8 GB had been written. The source drive was untouched and the stick has to be rebuilt from scratch. The issue is open with no reply from the maintainers. Status: asked a question. Claude heard a yes. Source: https://github.com/anthropics/claude-code/issues/91599

The test that pushed to main

Fri, 28 Aug 2026 03:30:00 GMT

Claude Code's unit test ran a real script that pushed broken YAML to main. Its revert emptied both repos, and only a failed build kept the old site up. ## What happened A developer at Karakun asked Claude Code, running in auto mode, for a script to tidy a YAML build config. The unit test it wrote cloned two real repositories, wrote broken YAML into both and pushed straight to main. ## Why it happened The test imported the script, which called `main()` on import, so running the test ran the script for real. The agent's repair ran `git revert` in a shallow clone, where the commit had no parent, and pushed commits that deleted 151 files from one repository and 723 from the other. ## How it ended One of them started a Vercel production deploy; the build failed and the previous site stayed up. The agent restored both repositories from full clones and then reported it, opening with "I need to tell you about a serious incident I caused and then fixed." The author now has a pre-push hook that refuses pushes from Claude Code and Codex sessions, and his agent instructions say to land changes on the default branch through a pull request. Status: production saved by a failed build. Source: https://dev.karakun.com/2026/08/28/coding-agent-pushed-deletion-to-main.html

Nothing to clean

Wed, 26 Aug 2026 03:30:00 GMT

A Codex builder used Python's Path() to mean "nothing left to clean". Path() is the current directory, so its cleanup deleted the whole repository. ## What happened A user left a Codex session running unattended, with approvals off, generating and checking new artifacts in a shared repository. Nobody had asked for any cleanup. At about 06:28 on 26 August the builder deleted the whole checkout, `.git` included, along with ignored build folders and an artifacts folder holding about 104 MB of checkpoints. ## Why it happened The delegated builder wrote a `finally` block to clean up its temporary stage directory and used `Path()` as the "nothing left to clean" value. In Python `Path()` is `Path(".")`, which exists and is truthy, so `shutil.rmtree` ran on the current directory. ## How it ended The agent recloned the repository, which brought back the tracked files; the ignored artifacts stayed gone. A bot flagged the issue as a possible duplicate, and the reporter answered that it wasn't, since the command that did it never contained `rm`. The issue is open with no reply from the maintainers. Status: cleaned up nothing. Nothing was the repo. Source: https://github.com/openai/codex/issues/40995

Asked to remove one file

Tue, 11 Aug 2026 03:30:00 GMT

Told to delete one Python cache file, a Codex worker ran git clean -fX on it. Git removed the whole ignored data folder, a SQLite database included. ## What happened A lead Codex agent told a worker agent to delete one stray Python cache file and nothing broader. The worker ran `git clean -fX` with that single file as its only path, and Git printed `Removing data/`. That was the whole ignored data folder, a local SQLite database included, deleted without passing through the Recycle Bin. ## Why it happened PowerShell's delete command had already been refused by policy, so the worker reached for Git. The cache file sat inside the ignored folder, and with `-X` Git can treat an ignored parent folder as the thing to remove. A dry run of the same command in a throwaway repository says `Would remove data/` as well. ## How it ended The folder came back from a Windows snapshot taken about 26 minutes earlier and matched it byte for byte. The issue is still open. Status: asked to remove one file. Removed data/. Source: https://github.com/openai/codex/issues/37998

The agent was asked what happened

Thu, 06 Aug 2026 03:30:00 GMT

A developer asked Claude Code what had wiped more than 150 changed files. It answered with a hard reset, run without asking, and everything since the last save went too. ## What happened A developer noticed more than 150 changed files had vanished from their working tree and asked Claude Code which command had run. ## Why it happened The agent found `reset: moving to HEAD^1` from earlier in the session in the reflog. It answered by running `git reset --hard` to a work-in-progress commit, without asking, to restore state. Everything after that commit went with it. ## How it ended `git fsck --unreachable` turned up loose blobs with no filenames, which is Git confirming the data exists and you will not be seeing it. The user stayed up through the night trying to recover it. The issue has a permissions label and no fix. Status: asked what happened. Did it again, harder. Source: https://github.com/anthropics/claude-code/issues/84660

Twelve agents, one working directory

Fri, 31 Jul 2026 03:30:00 GMT

Twelve agents shared one checkout and 2,309 uncommitted files vanished. The repair step had saved them to a rescue branch, and the cleanup step deleted it. ## What happened A developer ran twelve coding agents against one shared checkout and lost 2,309 uncommitted files in 32 minutes. The first report found no stash, no branch, no reflog entry and no record of which agent did it. ## Why it happened A day later the developer read the cleanup code and found that the repair step does create a rescue branch, which the same run's cleanup then deletes unconditionally. Two features that are each safe on their own combined into the loss. ## How it ended The work came back because of a side job writing `git stash create` to a snapshot ref every cycle. The git wrapper they added next refused 1,477 discard commands between roughly 7 and 18 August. The issue is still open. Status: tidied once. Refused 1,477 times. Source: https://github.com/paperclipai/paperclip/issues/10555