BuildPass Service Incident Report - Sept 19, 2024
Incident Summary
Between 6:58AM AEST and 7:22AM AEST on 19 September, BuildPass experienced a service disruption affecting our application. We apologise for any inconvenience this may have caused our valued customers.
Timeline
- 6:58AM AEST: Partial errors detected, resulting in degraded application performance for all Users
- 7:38AM AEST: Steps to improve system performance put in place resulting in improved performance for some Users
- 8:05 AM AEST: Services confirmed as fully restored for all Users
Impact
The incident resulted in a partial or total outage for some Users for up to an hour. It appears the outage was intermittent with Users reporting some pages loading as expected while others were failing. During this time, users may have experienced:
- Difficulty accessing the BuildPass admin application
- Slow response times
- Error messages when attempting to perform actions
- Workers being prompted to complete a "company induction" unnecessarily
Resolution
Our engineering team quickly identified and resolved the issue, restoring full-service functionality at 8.05 AM AEST. Company inductions completed by workers unnecessarily have been archived ensuring they can still be accessed by Builders while not appearing as requiring review.
Next Steps
Upon further investigation of the matter we have identified that the performance of our production database was the cause of this outage. We will be looking to make updates to the database to improve performance as well as migrating to a new database provider to improve load management during peak usage times.
As always, we remain committed to providing the highest level of service reliability to our customers.
If you have any questions or concerns regarding this incident, please don't hesitate to contact our support team on (03) 9917 4285.