How to Build Dynamic Forms with Large External Data Sources
By Rubicon engineering team · Published · Updated
The browser cannot hold your catalogue
The naive approach — loading every option into the page — falls over well before 100,000 rows. The fix is architectural: keep the list on the server, search it as the person types, and return one page of results at a time. The form stays fast because it never holds more than a screenful of options.
Version the data like you version the form
Reference data changes: equipment is retired, sites are renamed. If a form silently follows the latest list, a submission made last month can no longer be interpreted. Import lists as immutable versions and pin each published form to the version it was published with. Refreshing the list then never changes what a live form offers.
Cascade with explicit keys
Region → site → equipment chains need a stable key passed between levels, not a display label. Keep the key in a hidden field the platform reconstructs from the published definition — never trust a posted value — and validate the final choice against the pinned list version and its parent on the server. That is what stops a forged request from picking equipment that does not belong to the chosen site.
Measure the interaction, not the query
A database query that takes 5 ms can still feel slow if the round trip takes 300 ms. Measure typing-to-options in a real browser, cache result pages on the client for repeat searches, and be honest about what is left: our own live measurements are documented in the project roadmap, including the targets we have not met yet.
Practical takeaway
Large lookups are a data-engineering problem wearing a form's clothes. Treat the list as versioned data, search it server-side, cascade on keys, and validate on submit — the form takes care of itself.