Scripting Windows Apps
I've played around occasionally with doing win32/ole automation on my Windows system using Ruby, and I'm starting to see lots of potential uses for it.
As one example, a few months back I had 30-odd MS-Word documents that were all in more or less the same format, and I wanted to convert them to a particular XML format. Just yucky data-munging. The XML-export options for Word all resulted in really gross XML output. I could have written a complicated XSLT transfomer, but I really just wanted to do it with a script.
I wrote a quick Ruby script that did the job. Extracting the information from the Word document was by far the most trivial part of it; the rest was just XML munging using Ruby's built-in XML support (REXML).
I could have done this in Perl with the Perl/Win32 extensions, but I'm disliking Perl more and more these days; it's worth another blog entry at some point as to why. Suffice it to say that Ruby makes life a lot easier, even if you have far less experience with Ruby than with Perl.
You can also do Win32 scripting with Visual Basic, but unless it's improved a lot since I last looked at it, it's not a very powerful language, comparatively speaking.
You can do Win32 scripting with Python, and I even had it set up at one point, although the installation was nontrivial. (Ruby just comes with the win32 support as part of the standard distribution, so there simply was no installation.) But for the most part, Ruby does things more cleanly and consistently than Python, and it has all of the nice Perl features that make Perl (in some ways) such a joy to use, e.g. Perl's built-in regex syntax, and string interpolation.
So when it comes down to it, Ruby's just my preference for scripting.
Just for kickers, last night I wrote a script that will fetch the subscribers of any two Amazon email lists, and show you who's only on one of the two lists, or who's on both lists.
Because of the Isaac authentication requirement, using a straight http request is nontrivial. You can do it in various ways, but none of them seemed like they'd be as easy as scripting Internet Exploder to get the pages for me, since I already authenticate in IE every morning.
It turns out to be surprisingly fast - there's essentially no latency from Ruby or IE; the latency is all in fetching the list members on the server side (once a list has been fetched and cached, the script runs in about a second.)
To use this technique, you just need five lines of code:
# required for win32 support
require 'win32ole'# fire up an invisible Internet Explorer instance
ie = WIN32OLE.new('InternetExplorer.Application')
# send it off to some page (doesn't have to be internal)
ie.navigate("http://{internal-website}/your-desired-url")
# poll until page loads, per Microsoft documentation
sleep 1 while ie.readyState() != 4
# fetch the HTML as a string into 'html' variable
html = ie.document().documentElement.outerHTML
# this is optional (ie quits anyway when Ruby process ends)
ie.quit()
And now you have the HTML for the page in the "html" variable, so you can parse it or munge it with regexes or whatever.
The polling line is a bit silly, but works. If you hate it, there's an option to give your script with a Windows event queue and register for event-based notifications from IE. It didn't look very hard either, but in this case seemed like overkill.
Here's my quick-and-dirty Ruby script that gives you the diffs between two Amazon mailing lists, or with the "-s" option, shows you who's subscribed to both of the lists:
#!/usr/bin/ruby
# Shows diffs between 2 Amzn email lists by scraping the intranet.
# Operates by using IE as an engine; requires Ruby on Windows.
# You have to have authenticated with Isaac in IE before running this.
# Author: stevey, 9/7/2004
require 'win32ole'# pass -s to set to true => shows which people are on both lists
$intersect = falsedef get_members(list)
puts "looking up members for #{list}..." $ie.navigate("http://{internal-website}/email-list/search/#{list}")
# poll until page loads, per Microsoft documentation
sleep 1 while $ie.readyState != 4
html = $ie.document.documentElement.outerHTML
if html =~ /There are 0 lists that contain \"([^\"]+)\"/ puts "Email list not found: #{$1}" exit 0 end members = [] html.split(/\r\n/).each { |line|
members << $1 if line =~ /.+email-list\/expand-user[^>]+>([^<]+)</ } members enddef get_args if !ARGV.nil? && ARGV[0] == '-s' $intersect = !(ARGV.shift).nil?
end list1, list2 = ARGV if list1.nil? || list2.nil? then puts "Usage: email-diff [-s] list1 list2" puts " -s: show intersection (who's on both of the lists)" exit 0 end return list1, list2
end# main script
begin list1, list2 = get_args() $stdout.sync = true $ie = WIN32OLE.new('InternetExplorer.Application')
members1 = get_members(list1)
members2 = get_members(list2)
if $intersect i = members1 & members2 puts "People who are on both #{list1} and #{list2}:" i.each { |p| puts " #{p}" }
elsif members1 == members2
puts "The lists have the same members (#{members1.length} total):" puts members1.join('\n')
else puts "\nPeople in '#{list1}' but not in '#{list2}':" (members1 - members2).each { |p| puts " #{p}" }
puts "\nPeople in '#{list2}' but not in '#{list1}':" (members2 - members1).each { |p| puts " #{p}" }
endensure $ie.quit if !$ie.nil?
end
Example output:
(confidential info deleted)
Pretty useful. And the script is tiny, even with error handling, usage printing, comments, etc. So it's easy to modify.
Here's an even simpler one called "whosin" that just shows you who's on the specified mailing list, so you don't have to fire up a browser window and wait for the page to load and render.
The 'whosin' script:
#!/usr/bin/ruby
# Shows who's in the specified mailing list. Works by driving IE
# (on a Windows box) using the OLE interface.
# Author: stevey, 9/7/2004
require 'win32ole'def get_members(list)
puts "looking up members for #{list}..." $ie.navigate("http://{internal-website}/email-list/search/#{list}")
# poll until page loads, per Microsoft documentation
sleep 1 while $ie.readyState != 4
html = $ie.document.documentElement.outerHTML
if html =~ /There are 0 lists that contain \"([^\"]+)\"/ puts "Email list not found: #{$1}" exit 0 end members = [] html.split(/\r\n/).each { |line|
members << $1 if line =~ /.+email-list\/expand-user[^>]+>([^<]+)</ } members enddef get_args list = ARGV.shift
if list.nil?
puts "Usage: whosin <list>" exit 0 end return list.chomp
end# main script
begin $stdout.sync = true list = get_args $ie = WIN32OLE.new('InternetExplorer.Application')
members = get_members(list)
puts "'#{list}@amazon.com' has the following subscribers:" members.each { |s| puts " #{s}" }
ensure $ie.quit if !$ie.nil?
end
Notice that I've got a fair amount of copy & paste going on between this and 'email-diff'. Fair 'nuff. Following the Law of Threes, if I wind up doing one more of these scripts, I'll throw the common email-list-lookup stuff into a module.
Modules are as easy to make and use in Ruby as they are in Perl -- easier, in fact. And they're more powerful, if you care to take advantage of Ruby's mixin facility, which is a bit like multiple inheritance without most of the headaches. And you don't need to put that stupid "1;" that Perl requires at the end. Good Ole Perl. Sigh.
Anyway, now I have this little cheesy framework for doing all sorts of intranet-scraping tools. If I want a command-line Amazon phone tool on Windows, it's a 30-minute scripting job (if that). If I absolutely had to have a 'daily diff'... well... Anyway, there are lots of potential uses for intranet scraping.
I haven't really tried yet, but I can think of two potential uses for scripting on Windows. Good ones, that is.
The first is writing a script, or set of scripts, that sets up a Windows box from scratch, configuring it to my liking. Whenever I get a new Windows system, I spend a LOT of time configuring it, usually a full day. I install the Microsoft PowerTools TweakUI stuff. I change registry entries to swap the Caps-lock and Control keys on my keyboard (a must for Emacs power users). I change the backgrounds, and the color schemes, and configure the taskbar, and start menu. I download and install the same old software, and then I configure THAT, too.
It's no wonder I only upgrade my Windows systems once every year or two, if I can help it.
However, I could automate the majority of that with a Ruby/Win32 script. All I'd need to do on a new box is visit www.cygwin.com to download/install Cygwin (which is almost a 1-click process, or close to it), copy my script into my /bin dir, fire it up, and go away for a while. Joy.
The second big use I can see, although it's not as well-defined yet, is scripting Outlook, and *possibly* Exchange as well.
For instance, I might be able to write myself a little admin script that can automatically accept meeting invites that meet certain criteria, provided the script can access my inbox and calendar.
Or I might be able to do better automated manipulation and cleanup of my Outlook folders than what the GUI gives me. Or... who knows? All I know is that Outlook doesn't do a bunch of stuff I'd like it to do, and scripting might help alleviate that.
We all (hopefully) know how useful scripting can be on our Linux boxes. You can (and should!) automate away any tedious typing tasks; not only does it free you up to do more interesting and creative stuff, AND help prevent the ol' carpal tunnel syndrome, it also gives you practice at doing a kind of coding that you might not otherwise get a lot of exposure to -- and that can help improve your regular day-to-day coding.
But I'm not sure many Amazon folks (or Linux users in general) have had the Win32-scripting side of things really "sink in" yet. It's only just barely starting to sink in for me. But I have a feeling once I start thinking this way, I'll spot all sorts of automation tasks that I'd never thought to automate before.
Try it out! If you're dead-set on using Perl, the Perl/Win32 SDK is free, and you can use either Cygwin Perl or the native ActiveState Perl port. I'd personally recommend Cygwin, but the reasons for that go way beyond the scope of this blog entry.
If you want to learn more about Ruby, or just discuss it with other Amazon folks, hop on the ruby mailing list. The main website for Ruby is http://www.ruby-lang.org.. It has an HTML-ized (in color!) version of the Programming Ruby book by the Pragmatic Programmer guys, Dave Thomas and Andrew Hunt. Great book, great intro to Ruby, well worth a look.
Enjoy!
(Published Sep 8th 2004)
Comments
p.s. -- here are some URLs for learning more about how to do Windows automation with Ruby.
First, there's this project called "web testing with Ruby": http://www.clabs.org/wtr/ -- it's a set of Ruby modules specifically designed for driving Internet Explorer programmatically using the techniques from my blog post. The idea is to be able to QA web applications by simulating end user behavior from automated tests.
Ruby makes this so dang easy, it makes me wish I were a QA test engineer.
Next, the Ruby Application Archive (RAA, sort of like CPAN for Ruby) has http://raa.ruby-lang.org/project/ruby-win32/ -- contains some convenient wrappers for Win32 processes, setting registry entries, and other goodies. Here's a search string that shows a bunch of Win32/Ruby libraries:
http://raa.ruby-lang.org/cat.rhtml?category_major=Library;category_minor=Win32
The win32ole library that comes "stock" with Ruby is a fairly low-level interface. It's the one I've been using in my scripts. It basically gives you access to any Win32 API call, or any OLE interface exposed by Windows or a Windows application. But that doesn't mean it's super convenient. These libraries on RAA look like they'll be a huge help, providing clean wrappers for things like the Windows event loop, concurrency constructs, processes, registry, etc.
I'll post again on this topic next time I get a chance to do something with Win32/Ruby. I've been way too busy lately, unfortunately. Soon, I hope.
Posted by: Steve Yegge at September 18, 2004 07:30 PM