Many domains require fast, efficient incremental algorithms with no runtime overhead, such as in approximate inference algorithms for machine learning. Currently, such algorithms must be produced manually. We propose the use of program synthesis to automate this work, since it is easily extensible, the generated programs are fast, and there is no runtime overhead. We explain the algorithm on a simple example and provide a comparison of synthesis to other techniques that automate incrementalization.