We are inching closer and closer to completing the effort to rewrite the back end of the graph server. This has included a whole new schema and rewriting all the scripts that interact with that schema. Due to the major differences between the old and new schemas data migration isn’t going to be easy. While we do plan on moving some data over (say around interesting branch points, or for retired branches that we are no longer actively testing), we figured that we would go for a plan where we would ease into use of the new database. This means that for the next few weeks, possibly a month, Talos boxes are going to be sending data first to the graphs.mozilla.org and then immediately sending the same results to graphs-new.mozilla.org. This should in no way affect the numbers collected by Talos, or impact the cycle time of Talos machines. This gives us a few benefits:
- Stress test the new graph server. We’ve had a staging version of the graph server with the new schema up and running for a while, but it only has 5 or 6 staging Talos boxes reporting to it. We need to see what happens when 90 boxes try to report results all at once.
- A good chance to pre-populate the new graph server db. When we had discussions about just migrating all data in the old db and forcing a switch over to the new graph server all in one shot we ended up talking on the order of 24-36 hours (or more) to convert and transfer all the data. Just doing double send for a while is going to be far less intrusive and will let us work out the remaining data migration issues under a more reasonable time frame.
The main change that developers who monitor the various waterfalls will notice is that Talos columns will have the standard links to graphs followed by a second set of graph links which will be to the new graph server. Expect this change to take place by the end of this week. If this causes any undue confusion, or you find bugs in the graphs-new.mozilla.org feel free to drop by #perfomatic to provide feedback.