Not knowing about what would be required to model these full representations, is it just a typical "too much data, not enough data scientists" problem, or are there fundamental research questions to be answered here? For that matter, does the SEC even employ or contract/grant big-data researchers for these types of problems? It seems like it's exactly the type of problem that a government research grant should be intended to solve!
Defining a swap: the OTC market is complex and there is no standard representation for many of the products - this makes reconciling data submitted by each of the firms complex. A simple interest rate swap report could contain up to 1,000 elements to fully describe a trade.
Volume: the 2 major OTC reporting regimes, Dodd-Frank and ESMA, both generate tens of millions of trade reports daily. This data needs to be correct; needs to be both ingested and retrieved quickly and needs to support complex post-reporting processing and queries.
Complex workflow: each swap undergoes many changes during it's lifetime, which complicates the workflow and data for reporting. For example, you may trade a block CDS, which is split into 20 allocations, some of which are new trades and some that alter existing trades (step ins; step outs; terminations; reductions...). Each of the allocations may require different reporting treatments.
And yes, the 'too much data, not enough scientists' problem with interpreting the submissions. Bear in mind that the SEC reporting hasn't started yet - that's 2015...
