Sourcehut

Restart completed. Again, note that some upstream connections may take a while to be re-established due to rate limits.

(12:05 UTC — Apr 3)

Planned bouncer upgrade is scheduled for Thursday, April 3rd. On Thursday at 12:00 UTC we will be rolling out an upstream update for soju, the software that powers the chat.sr.ht bouncer. The bouncer will be restarted, causing everyone to be disconnected. Most clients will reconnect automatically. Note that some upstream connections may take a while to be re-established due to rate limits.

We have deployed Anubis to git.sr.ht. After some internal discussions we have ultimately decided that the best course of action to protect git.sr.ht from LLM crawlers is to deploy Anubis. This software presents some users with a proof-of-work challenge which is solved by the user’s browser with JavaScript.

This challenge is automatically bypassed for logged-in users. If your browser does not support JavaScript (or you do not wish to enable it for any other reason), log in at meta.sr.ht to circumvent it.

Maintenance completed. Access to all services is restored. (11:40 UTC — Feb 11)

Maintenance will take longer as expected. Due to unforeseen circumstances, the maintenance will take longer than anticipated. We are working to resolve the issue as quickly as possible. (11:00 UTC — Feb 11)

Maintenance is underway. The maintenance window is now open. (10:00 UTC — Feb 11)

Planned maintenance on February 11th will cause intermittent outages: Planned maintenance starting at 10AM UTC will affect all services, causing intermittent outages that are expected to last between 15 and 30 minutes at most. Notably, chat.sr.ht will be rebooted, causing everyone to be disconnected from the bouncer. Most clients will reconnect automatically. The maintenance is expected to take no more than an hour.

SourceHut continues to face disruptions due to aggressive LLM crawlers. We are continuously working to deploy mitigations, instead of doing any of the other numerous things we have to do, like develop new features, upgrade our infrastructure, or respond to support emails in a timely manner.

We have deployed a number of mitigations which are keeping the problem contained for now. However, some of our mitigations may impact end-users.

If you are unable to use the SourceHut web UI:

Mitigations completed: We believe that we have the attack under control and that there is no need for end-user impacting mitigations to remain in place. (09:00 UTC — Jan 3)

Attack still ongoing, but under control: Good morning. The attack is still ongoing, but we have been refining our mitigations this morning. We addressed some cases which were causing git and hg to crash under load, and narrowed our mitigations to better characterize the load. All of our mitigations have now been disabled for logged-in users, reducing the impact on legitimate traffic – if you hit an error page, log in and try again.

Mitigations complete. We have finished mitigating the attack and have removed user-impacting mitigations. (17:00 UTC — Dec 4)

Attack partially mitigated. We have partially mitigated the attack to restore service to git.sr.ht, but many features of the web UI are currently disabled to reduce load. (12:53 UTC — Dec 4)

A distributed denial of service (DDoS) attack on git.sr.ht is underway: A denial of service attack against git.sr.ht is currently underway. We are working on deploying a mitigation. (12:42 UTC — Dec 4)

Maintenance work is completed. All services should be operating normally. Please let us know if you notice any lingering problems. (13:26 UTC — Nov 25)

The maintenance window is now open. (13:00 UTC — Nov 25)

Maintenance window rescheduled for 13:00 UTC. We have advanced the schedule by one hour to 13:00 UTC. (08:00 UTC — Nov 25)

Planned maintenance is scheduled for Monday, November 25th. On Monday at 13:00 UTC we will be rolling out some changes which will cause spurious errors and issues across services for the duration of the roll-out. Notably, chat.sr.ht will be rebooted, causing everyone to be disconnected from the bouncer. Most clients will reconnect automatically. The maintenance is expected to take no more than an hour.

Upstream network issues appear resolved. We still know little of the cause but we know it broadly affected numerous providers in our service area. (14:00 UTC — Oct 30)

Upstream network issues are causing intermittent availability problems for SourceHut. An issue with our upstream network provider has affected user access to SourceHut. We do not have any further details or an ETA on a solution at this time. We expect the network to be intermittent until the upstream issue is resolved. (11:30 UTC — Oct 30)

Some users are having problems with chat.sr.ht. There are some known issues with chat.sr.ht which are preventing a small number of users from accessing the service. The problem is known and being investigated, but it is complex and difficult to fully understand and address. We request your patience as it may require several days to identify a complete solution. (08:00 UTC — Aug 14)

All planned maintenance has been completed. The maintenance window is now closed. Please notify support if you encounter any unexpected issues.

The maintenance window is now open. We will be opening our maintenance window in the next few minutes and begin rolling out upgrades.

A planned maintenance window is scheduled for August 5th. The maintenance window originally scheduled for August 1st has been rescheduled for Monday, August 5th at 11:00 UTC. This is expected to cause intermittent partial outages between 15 and 30 minutes affecting all services. Follow this page or #sr.ht on IRC for updates.