Get started Features Tutorials Blog Research
August 10, 2022

VowpalWabbit 9.3.0 Release Notes

author's avatar
Jack Gerrits

Experimental support has been added for merging several compatible VW models. This has similarities to allreduce which allowed for several VW nodes to synchronize state to implement parallel learning. Model merging also works by taking several VW models and combining them to produce one which incorporates them all. However, unlike allreduce it does not do so by coordinating several parallel VW nodes via network. It does so with the model files offline. Model merging has been designed to be extensible to each reduction to incorporate a more correct representation of the state of a model and so should produce more correct results.

For now just GD and CB_ADF reductions have had merge operations defined. If a reduction defines a save_load function, it signals that it has state that is required for for inference or learning. Therefore, a merge operation is required for each such reduction. If the merge API is used and a reduction with save_load but no merge is encountered then a warning is produced. allreduce only ever really implemented the GD layer of merging though so the thinking here is that it is the same or better as the old way.

It has APIs exposed in Python, Java, C++ and a command line tool vw-merge.

More details can be found on the wiki page.

Thank you

Thank you to all contributors:



Other changes