Capillaries: notes
2024-08-10
Capillaries: scale up vs scale out
This blog post continues the scalability discussion started earlier. There are a couple of technical changes this time:
- Cassandra nodes use local NVME drives for storage (RAM drives were used in past experiments)
- Daemon instances are now x4 less powerful than Cassandra nodes (previously, they were 2x less powerful)
- Graviton-based instances are used (previously, AMD instances were used)
2024-03-10
Capillaries: analyze how Fannie Mae transfers credit risk from taxpayers to investors - mortgage by mortgage, payment by payment
Since 2013, following the "Big Short," Fannie Mae has been issuing mortgage-backed bonds as part of its "Connecticut Avenue Securities" (CAS) program. This typically occurs several times a year in the form of so-called "deals." For each deal, Fannie Mae selects a pool of qualified mortgage loans and issues bonds that enable investors to receive a portion of the interest payments from the mortgages in that pool. In return, investors are exposed to the risks associated with those mortgages.
For each deal, Fannie Mae publishes data about the mortgage pool. The data comes in the form of a dataset that can be downloaded for free from Fannie Mae's website. Read more...
2023-11-15
Capillaries: ARK portfolio performance calculation at (slightly bigger) scale
At the end of the day, using any piece of technology is about saving money. This post attempts to estimate the potential costs of implementing a Capillaries-based solution for portfolio performance calculation. It gives a realistic estimate of the time and money an organization may need to allocate to produce portfolio performance results from raw data files. Read more...
2023-04-08
Use Capillaries to calculate ARK portfolio performance
Recently, a new Capillaries integration test was added: Portfolio performance calculation. It mimics the process performed by an investment bank at the end of each quarter or year: for each individual portfolio, combined return rate is calculated for a specific period. This test uses data for six portfolios maintained by ARK, and calculates yearly performance for the whole portfolio and for each sector using time-weighted annualized return (TWR) formula.
What's so cool about it? The test performs calculations for six portfolios only, but the process can be easily scaled out to tens or hundreds of thousands of portfolios. And this is exactly the challenge that investment bank IT departments are facing on a regular basis: provide each client (there may be hundreds of thousands of them) with a personalized rate of return, and do that, say, overnight. Read more...
2023-02-20
First look at Capillaries UI
Below are two scraped snapshots of Capillaries UI that give the idea about the state of the test environment. Click any of the screenshots and navigate through status screens - all blue elements are clickable, buttons are disabled (button tooltips are working though).
Second snapshot also contains status information about other integration test runs: lookup quicktest (smaller number of orders and order items), py_calc_quicktest (performs Python formula calculations) and tag_and_denormalize_quicktest (tags data records using regular expressions and calculates tag totals). Read more...