Getting the Risk Calc libraries compiling on some compiler

RAMAS Risk Calc (http://www.ramas.com/riskcalc.htm) was written in C++ in Borland’s C++Builder. Because of compiler updates and unsupported libraries, the code no longer compiles in the current version of C++Builder. Although various changes over the last several years could have been made to the Risk Calc code to improve its functionality (and some actually have been made to the uncompilable code), virtually none have been delivered to users because of this problem. We want to get the software compiling again so that a new version of the code can be released. It may also be useful to create relatively independent libraries that capture Risk Calc’s functionality so that it may be tied into other programs as well.

There may be a couple or several issues that are preventing compilation, but it seems like most of them are related to the interface rather than the calculation engine itself. It seems that the latest versions of C++Builder do not support some of the libraries or structures that were used to construct the Risk Calc “Listener” which is the interface for the program. I have the sense that these issues are fairly small in the greater scheme of things, but they have nevertheless been insurmountable for me.

Foreseeing similar problems into the future, we might want to escape from the proprietary C++Builder altogether and migrate to another C++ platform that might be more stable or at least more navigable. Exploring that possibility, I’ve had some discussions last year with our employee James Mickley, which are displayed below.

Last month I asked James to update and summarize his thoughts, and he sent two emails:

Risk Calc is a calculator that consists of multiple uncertainty propagation libraries (interval, fuzzy and probability bounds), units propagation, graphical display, and a scripting language. It includes libraries for

Ancillary mathematical functions,

Interval and fuzzy arithmetic,

Probability bounds library (downs.cpp),

Arithmetic and units handling (arithmic.cpp, value.cpp, etc.),

Reading Microsoft Excel files,

Parser-lexer (including the Yacc and Lex specifications), and

Interface for textual input and output for the Listener, help, & graphical displays.

There are probably a couple dozen files. Like all multi-year projects with many programmers, the code is fairly complicated and idiosyncratic. It’s not awful, but it’s not stellar either. Fortunately, however, most of this code is appropriately independent of the user interface. For instance, error messages are given with wrappers, etc.

UPDATE:

There have been some recent emails on the prospect of engaging Bill Root on this effort.

Scott emailed Bill Root:

I'm not sure whether I've mentioned to you that we cannot actually compile the current version of Risk Calc except on my old machine with the giant monitor and the 2002 (or maybe it's 1997) version of Windows. I think the problem is that some libraries we used are no longer available in the latest version of C++ Builder. Maybe I did mention this to you, as I seem to remember our talking about it. Remind me if we already came to conclusions I've forgotten.

Anyway, we really need to be able to compile the stupid thing again. And I think we might want to take this opportunity to separate the calculation engine from the interface (which of course should have been done from the start...and, I think, they are as separated as Ruediger could make them). I've had Jimmie Goode working on this, but his progress has been glacial in a way that only a graduate student can conceive of.

I guess it's time to turn the problem over to you. It seems to me there are several parts to this puzzle:

We want to decide on a better development environment. Discussions with Jimmie and James Mickley and perhaps you have led me to believe that we should be using one of the public compilers simply because they don't change so damn often. Jimmie and I agreed on a C++ compiler to use, although I've forgotten which one it was. (I can reconstruct the decision.)

We want to convert the code in the calculation modules to the strictures of the chosen compiler. It seems to me that this should be pretty easy once you get going. The resulting libraries should make no interface demands at all, and should be suitable for incorporation into other programs. Again, Jimmie has done some of this. I guess he has DOWNS.CPP working (which is really the heart of the program's calculations).

But then to revivify Risk Calc itself, we'll need to reconstitute the interface, or at least much of it. This may be moderately complex. I guess to be smart about this, we'll want to create a library of wrappers for all the interface functions. Do you agree? This will enable us to bring functionality on line sequentially, rather than having to finish everything before anything works. You'll remember that, in addition to reading and writing from the Listener and Display windows, and the standard panoply of file, editing, page setup, window tiling, printer dialog and help access, we also support editing for ascii/RFT files and tables (a special format of ascii file), a syntax wizard, an options dialog, conversion from Excel, and importing variables. There are also modest right-click popup menus for the Listener and Display windows. (I would like to be able to access all these features programmatically as well as by mouse clicks too, but that's maybe for another discussion.) I think most everything is pretty straightforward, except maybe the interactions with Excel, but you implemented those yourself, so they shouldn't be scary. I think that, if we do this right, it should be possible to get Risk Calc versions for Windows, but also Mac and Unix, and perhaps even mobile apps.

Are you available for these tasks or parts of them? I think we need to bite the bullet and get you to do it. Can you estimate any thing about what you think it'll take in terms of your dollars? To make a better estimate, I can perhaps send you what Jimmie has done so far. I can call you phone if you'd like to chat a bit more about this. If so, let me know a time to call.

By the way, do you know anything about Antlr which replaces Lex/Yacc? If not, nevermind.

Bill responded to Scott:

You have spoken to me about the problems building Risk Calc (in May 2010, at least). At the time you asked about migrating it to the newest version of C++Builder. With an hour of effort I gave the wonderfully helpful guesstimate of 1 week to a month of effort, plus the cost of the newest C++Builder. (After converting RAMAS GIS to the latest version of Delphi, I now consider that estimate low.) The task you're suggesting is substantially greater, probably around an order of magnitude more so.

You have some good ideas, in my opinion. I too am tired of the annual Delphi/C++Builder upgrades. I skip most of the Delphi upgrades, both for cost and conversion effort reasons. Using a stable open source or free C++ compiler would probably mean fewer changes, or at least at a slower pace. (Note, however, than some open source projects change as fast as or more frequently than commercial products!) I suspect extracting the Risk Calc calculation code and converting to another compiler would still be a significant effort, especially if the new compiler uses Unicode strings. Strings and C++ have never gotten along well (too many kinds of strings), and Unicode strings are a whole additional source of problems. If the task was easy Jimmie would have more code working.

One advantage of separating the calculation code is the ability to make meaningful unit tests. Each module and each operator can be tested standalone in an automated fashion. This would make progress apparent and make it possible to to estimate the task of converting the rest of the calculation code.

Building a new, similar UI (user interface) in another product might be easier than converting the calculation code, depending on the environment chosen and the final requirements. I have found Delphi XE2 to be quite productive for the UI of RAMAS IRM. Java would be slower (for me, anyway), as would C++Builder or probably anything else. A substantial effort may be the glue code so the UI can use the C++ calculation engine. Using different tools for each piece (calculation engine and UI) most likely means a simple function interface between the two (probably in a DLL, for Windows, anyway), and converting objects to a simple form (marshalling) on one side of the interface and then to the object form on the other side (unmarshalling), with different code defining the objects on each side, since the environments differ. An alleviating factor is that the calculation engine may only use one type of object (I don't recall exactly).

The Excel connection is possibly the hardest part. I remember the days of working on the Excel interface as like pulling teeth. I'm not sure I could be convinced to work on that again. In the interest of multiplatform support, perhaps you can accept a looser coupling, such as CSV files or data on the clipboard.

As far as multiplatform support goes, plain C++ is very portable (supposedly). If the calculation code was compiled by a compiler supported on all your desired platforms, then even better. The user interface is more challenging, as you know. Java is on many platforms, but not all: it's completely disallowed on iPad & iPhone, and Android, while having a Java heritage, doesn't run Java programs as far as I know. Delphi XE2 programs can be compiled for Windows and Mac OS X. It has limited iPhone support, but goes through hoops for it. Embarcadero promises future mobile support, but it sounds like it will be a separate product. HTML5 and JavaScript may be the greatest common denominator, but calling C++ from JavaScript is very difficult and limited, although it sounds like PhoneGap (a mobile development platform) might have a way to do it. (Of course, a web page could call the web server to do calculations, but that changes the model significantly.)

But then to revivify Risk Calc itself, we'll need to reconstitute the interface, or at least much of it. This may be moderately complex. I guess to be smart about this, we'll want to create a library of wrappers for all the interface functions. Do you agree? This will enable us to bring functionality on line sequentially, rather than having to finish everything before anything works.

When you say "interface functions", do you mean the calculation interface? If so, I agree. I don't know whether the interface would be in an object library, a DLL, or something else. It depends on how the user interface is developed. In some cases, a text file / process interface might be required, although that would be unwieldy.

I'm overbooked at least until early November, and even at that time I would decline to offer an estimate. There are too many unknowns. I would be happy to do some development work after early to mid November on parts with a limited and well-defined scope. The progress would help you estimate the total effort required.

James Mickley replied to a forward of Bill's response:

I'm guessing you want me to comment on this???

Sounds like Bill thinks it's a lot of work, I'm inclined to agree having looked at it, pretty sure Jimmy would too. Your interface is intimately attached to the rest of things as far as I could figure, but who knows. Bill's got an efficiency advantage because he knows this stuff more, but I suppose I might be cheaper. I'm not salivating over this or anything though. I think it would take me several months at a minimum at my 10-15 hrs/week, and maybe longer given Bill's appraisal.

The interface shouldn't be the worst part for sure and if you use QT or GLADE as we talked about a while ago (http://qt-project.org/, http://glade.gnome.org/), then you could build one interface that would work on mac, windows, and linux. And they're free.

The Excel part may have gotten easier. Excel now uses an "open" storage format that's basically a zip file with XML files inside. Those XML files as you might remember from last week are text files similar to HTML and are pretty easy to parse. Still it's probably a bit of a headache unless you can find a library that already does it, which might be fairly easy. I'd at least look there first.

The iPhone is a completely closed platform unless it's jailbroken. You have to use Apple's Objective C to code apps, and nothing else if you want to be in the App Store. There have been some attempts at pre-compilers which allow you to write in one language that's then translated to Objective C, but I don't know that they amount to much. Bill seems to know a bit about this. Android on the other hand (Google's counter to iPhone) is java-based and much more open.

I think the ONLY common denominator is a web app using HTML5 + Javascript. And that has its own problems as Bill mentions, in addition to some issues of browser compliance and general clunkiness. It would be much easier to write a web app that called ESTPbox.R for the phones than it would be to use riskcalc code I think.

We need to, somehow, get the Risk Calc code compiling again, whether on a Borland compiler or some other platform. The effort should perhaps start with a review and assessment of which platform(s) should be used. It might be reasonable also to look at what Jimmie Goode's progress has been. There is a burning need for this.