Practical Distributed Computation

The distributed computing applications that have emerged recently – Skype, BitTorrent, Video On Demand, and others – while vertically limited and sometime controversial, are among the most exciting applications the Internet has produced.  PLATFORMedia has developed a proprietary peer–to–peer application to enable the distributed processing of time–intensive computations across a company’s own Intranet, or for use in a community of PC users with common interests and a shared desire to increase their individual processing power.

The emphasis in our peer–to–peer solution is on the practical implementation of parallel processing.  Thus, the application (the Compute Engine) can be downloaded and installed on any standard PC. It communicates using standard internet protocols, and can interface easily to Microsoft Office applications – notably Microsoft Excel.

I : Problem : Computing Intensive Tasks

Examples of problems well–suited to our practical distributed computation solution include financial modeling and Monte–Carlo simulations.  In addition, there are a myriad of other business–critical problems that require more computation than is available in reasonable time on a standard PC.  There are also many optimization problems that have been considered unfeasible up to now such as the optimization of designs, distribution center placement, and scheduling and packing problems.  The combination of evolutionary methods with distributed processing is particularly powerful and, since it is an area of specialist expertise for PLATFORMedia , we have developed software front–ends that specifically facilitate working with these methods.

II : Solution : PLATFORMedia

Technology

PLATFORMedia has developed the complete peer–to–peer architecture for the practical implementation of distributed computation to solve real–life problems without requiring advanced programming skills.

Typical order of events :

• A client of the company has a problem that requires brute–force computation. Typically, this is expressed as a large Excel workbook.

• The problem is split into sub–problems, where each sub–problem is a copy of the main workbook using a different selection of input parameters, e.g., ‘ScenarioNumber’, ‘RandomSeed’, ‘DefaultRateAssumption’, ‘YieldCurve’, etc. Any number of inputs are allowed.

• A list of these sub–problems is created on a copy of our task manager spreadsheet.

• The list is ‘sent’ to the client’s local Compute Engine – installed on his PC, as well as other PCs within the same corporation.

• By using the spare computer processing cycles on each of the machines on the local LAN, the list of tasks is quickly distributed, each part executing on a different machine, in the background.

• The resultant cells on each spreadsheet are sent back to the client’s local machine and appear next to the list of tasks once they are completed. Any number of cells can be used as outputs, such as ‘VaR’, ‘MeanPV’, ‘TotalCost’, ‘MaximumDrawDown’, ‘SavingsMade’, ‘ProjectedTotalValue’, etc…

The end result is that the client task has been executed in parallel, allowing the problem to be further refined, and action taken far more quickly and effectively than possible before.  And since each peer–to–peer node is ‘equal’, the same powerful environment is available to every person on the corporate LAN with the Compute Engine installed.

Whilst there are many applications for this technology to ‘ordinary’ hard computing tasks, PLATFORMedia has specialist expertise in evolutionary methods, the value of which is now becoming more widely recognized.  We have pre–built front–end interfaces that contain best–of–breed technology to manage these optimization methods that can be distributed extremely efficiently.

20 October 2009