Chain of Fools : Upgrading through every version of windows

Every now and again some of us get reminders of how old we are. This excellent video on upgrading a system from MS-DOS 5.0 through all the Windows versions (except ME!) to Windows 7 is truly frightening as I remember doing all of this! The amazing thing is that this was even possible …

 

Do you remember any of this?

Many hands make light work

The last posting on collections, whilst being a bit on the long side was fairly straightforward to follow. Today’s posting is a little more, umm, challenging … The reason is I really want to show you what’s possible in the .NET Framework 4.0 that means you will be able to make your apps scream along on modern, multi-cored processors with minimal effort.

First off, a little theory. Most programming languages provide the ability to execute external functionality through transferring control to code referenced by a pointer. Modern COBOL in fact uses procedure pointers to provide this functionality (it just wasn’t possible in OS/VS COBOL). The problem with this type of transfer of control is that it is not safe: you can’t guarantee that you are passing the correct parameters or will receive the correct data back. The .NET Framework gets around this problem through the use of Delegates. Delegates are a type safe way of invoking functionality. This invocation may be synchronous or asynchronous. A delegate describes what some function looks like without defining it. In order to use a delegate you must go through three steps:

  1. define the delegate or method signature
  2. instantiate the delegate
  3. invoke the delegate

In this particular posting, I have to admit we’ll be going through this (to use the vernacular), arse about face. The only reason for this is I had a bit of fun writing this particular bit of code and I want to get it out there and then get on to explaining delegates better!

In this example we’re going to use a generic delegate already defined in the .NET Framework: System.Action. If we look at the code for the new DirectoryCollection (nothing else was modified), I’ve highlighted the new code and commented out the replaced code: 

class-id DirectoryUtility.DirectoryCollection  
  inherits type System.Collections.Generic.List[type System.IO.FileInfo].

working-storage section.

01  DirectoryName   string as "DirectoryName" public property with no set.
   
method-id New.
local-storage section.

01  dirInfo         type System.IO.DirectoryInfo.
01  fiArray         type System.IO.FileInfo[].
01  fInfo           type System.IO.FileInfo.
    
procedure division using by value directoryName as string.

    Set DirectoryName to directoryName.
    Set dirInfo to new type System.IO.DirectoryInfo(DirectoryName).          
    Set fiArray to dirInfo::GetFiles("*.cbl").
    
    Perform varying fInfo through fiArray
        Invoke self::Add(fInfo)
    End-Perform.

    goback.
end method.

method-id Compile public.

local-storage section.

*>01  fInfo           type System.IO.FileInfo.
01 actionDelegate   type System.Action[type System.IO.FileInfo].     
       
procedure division.

    Invoke type DirectoryUtility.CobolUtilities::Initialise().
*>   Perform varying fInfo through self
*>       Invoke type DirectoryUtility.CobolUtilities::
*>                   CompileProgram(
*>                       fInfo::FullName, 
*>                       fInfo::DirectoryName
*>                                 )
*>   End-Perform.          

Set actionDelegate to new type Action[type System.IO.FileInfo]
(
delegate using fInfo as type
System.IO.FileInfo
invoke type DirectoryUtility.CobolUtilities::

                            CompileProgram(
                                fInfo::FullName, 
                                fInfo::DirectoryName
                                          )
             end-delegate
        ).       
 
    Invoke type System.Threading.Tasks.Parallel::ForEach
                    (self, actionDelegate).
                     
    goback.
    
end method.

end class.

Firstly, I define actionDelegate as a System.Action generic delegate of type System.IO.FileInfo. Then I instantiate actionDelegate using the anonymous delegate syntax (the delegate … end-delegate stuff) with the identical code that had been wrapped previously in the Perform varying … End-Perform. I can then invoke the delegate for each System.IO.FileInfo object within the collection using System.Threading.Tasks.Parallel’s ForEach method. This function, part of the Task Parallel Library of .NET Framework 4.0 knows about your hardware and is  able to schedule processing of tasks in parallel to optimise the usage of that hardware. So is it worth it? You be the judge:

 

SNAGHTMLba62613

Look at the duration there: 14 minutes 22 seconds (or 862 seconds). Sequentially, this took 46 minutes 18 seconds (or 2778 seconds). That’s a reduction in duration of 69%. This program was running whilst I was editing this posting, had several Internet Explorer windows open, Outlook, Visual Studio and lord knows what else, at no time losing responsiveness.  Look at Task Manager for some of those programs below. Notice the number of compiler processes running at the top …

image

 

Now I do admit my laptop is a quad core, but that’s what those cores are for, isn’t it? The System.Threading.Tasks functionality drives those cores hard without overwhelming them. Check out the Performance tab on Task Manager during the running of the program. Notice the workout each CPU is getting.

image

The program isn’t production strength and could definitely do with some polishing (the displays come out at a different point to the compilation output due to the fact that the threads are running concurrently), but still not bad for an hour’s work (in total – including the previous post’s code)…

The code can be found here:

http://cid-29bd479bafba9591.office.live.com/embedicon.aspx/COBOL%20School/Multi-threaded%20Cobol%20Directory%20Collection.zip

Welcome to the worker’s collective

The code has been written for a while but I’ve been struggling with writer’s block on what to say about it for a couple of days … Code comes easy but words fail me? Sounds like I might end up being condemned to the back room …

This post is a bit of an introduction to a feature of the .NET Framework called Generic Collections. They can be simply described as COBOL arrays on steroids plus a whole lot more. Just about any way you can group objects has already been written: Lists, Dictionaries, Stacks and many more.

Within the .NET Framework, generics provide a mechanism for writing classes using placeholders rather than specific types so that the developer may write type-safe general classes. Most of the generic classes you are ever likely to need are already written and are available as part of the Framework. Since generics were introduced in the .NET Framework 2.0 in early 2006, I haven’t written a single generic class in anger (although I have played a little in order to understand them).

Once upon a time it was a time-consuming task to write a collection doing complex things like getting enumerators correct. Those days are gone, now we can simply inherit from the appropriate generic collection, add a little specialised behaviour and plough on with the real work. Let me show you … We’re writing a little application to compile all the COBOL programs in a directory so let’s create a collection of directory entries that we may iterate through to compile everything.

First, here’s our solution structure:

image

There are two projects in the solution: CompilePrograms a simple COBOL Console application that references the second project, DirectoryUtility, a COBOL Class Library. I try to put anything I might reuse into class libraries that I might reference in simple top level programs that may define a user interface. With little effort I would be able to create a Windows Forms interface, a Windows Presentation Framework or even a Silverlight interface to the same functionality.

The core of the application is in:

class-id DirectoryUtility.DirectoryCollection  
   inherits type System.Collections.Generic.List[type System.IO.FileInfo].

working-storage section.

01  DirectoryName   string as "DirectoryName" public property with no set.  

method-id New.
local-storage section.

01  dirInfo         type System.IO.DirectoryInfo.
01  fiArray         type System.IO.FileInfo[].
01  fInfo           type System.IO.FileInfo.
    
procedure division using by value directoryName as string.

    Set DirectoryName to directoryName.
    Set dirInfo to new type System.IO.DirectoryInfo(DirectoryName).          
    Set fiArray to dirInfo::GetFiles("*.cbl").
    
    Perform varying fInfo through fiArray
        Invoke self::Add(fInfo)
    End-Perform.

    goback.
end method.

method-id Compile public.

local-storage section.

01  fInfo           type System.IO.FileInfo.
       
procedure division.

    Invoke type DirectoryUtility.CobolUtilities::Initialise().

    Perform varying fInfo through self
        Invoke type DirectoryUtility.CobolUtilities::
                    CompileProgram(fInfo::FullName, fInfo::DirectoryName)
    End-Perform.

    goback.
    
end method.

end class.

The first thing to notice is that in the class definition I’ve inherited from  System.Collections.Generic.List[type System.IO.FileInfo]. This class will be a strongly typed collection of System.IO.FileInfo objects but could as easily be a Dictionary (for fast keyed access) or a Stack (to push and pull objects from). I’ve overridden the new method to populate the list when you create the collection by merely passing a string containing the directory name (I probably should generate an exception if the directory doesn’t exist, but you get the idea). I’ve also created a new method Compile, that will invoke a utility class to initialise the Micro Focus Visual COBOL environment and invoke the compiler for every item in the collection. Note the syntax to iterate through the collection:

Perform varying fInfo through self

..

End-Perform.

We’re processing every fInfo (System.IO.FileInfo) in self , or the collection itself. No indexes to maintain or danger of going beyond the bounds. I’d suggest that in the .NET world no COBOL program should ever create a traditional array ever again. This is simple, powerful stuff.

You probably want to check out the CobolUtilities class out of curiosity:

class-id DirectoryUtility.CobolUtilities public.

static.

working-storage section.

method-id Initialise public.
local-storage section.

78  EnvLocn 
value "SOFTWARE\Wow6432Node\Micro Focus\Visual COBOL\1.2\COBOL\Environment".
01  copyDir     string.
01  settings    type DirectoryUtility.Properties.Settings.
01  regKey      type Microsoft.Win32.RegistryKey.
01  subKey      type Microsoft.Win32.RegistryKey.
01  keys        string occurs any.
01  aKey        string.
01  aValue      string.
       
procedure division.

    Set settings to new type DirectoryUtility.Properties.Settings.
    Set copyDir  to settings::CopyDirectories.
    Invoke self::SetVariable("COBCPY", copyDir).
    
    Set regKey      to type Microsoft.Win32.Registry::LocalMachine.
    Set subKey      to regKey::OpenSubKey(EnvLocn).
    
    If  subKey not equal null
        Set keys to subKey::GetValueNames() 
        Perform varying aKey through keys
            Set aValue to subKey::GetValue(aKey)
            Invoke self::SetVariable(aKey, aValue)
        End-Perform
    End-If.          
    
    goback.
    
end method.

method-id CompileProgram public.

local-storage section.

78  compileLine     value quote & "{0}" & quote 
                          & " ILGEN NOQUERY RAWLIST;".     
01  compileOptions  string.
01  pInfo           type System.Diagnostics.ProcessStartInfo.     
01  aProcess        type System.Diagnostics.Process.  
01  compilerOutput  string.

procedure division using by value programName       as string
                                  programDirectory  as string.

        Display "Compiling " programName.
        Set compileOptions to type 
                           System.String::Format(compileLine, programName)
        Set pInfo to new System.Diagnostics.ProcessStartInfo("cobol.exe")
        Set pInfo::Arguments to compileOptions
        Set pInfo::WorkingDirectory to programDirectory
        Set pInfo::UseShellExecute to false
        Set pInfo::RedirectStandardOutput to true
        Set aProcess to new System.Diagnostics.Process
        Set aProcess::StartInfo to pInfo
        Invoke aProcess::Start()
        Set compilerOutput to aProcess::StandardOutput::ReadToEnd()
        Invoke aProcess::WaitForExit()
        Display compilerOutput

    goback.
    
end method.

method-id SetVariable private static.

local-storage section.

procedure division using by value   envName as string
                                    envValue as string.
                                    
    Invoke type 
           System.Environment::SetEnvironmentVariable(envName, envValue).       

    goback.
    
end method.
end static.
end class.

First off, what the heck am I doing in the Initialise method? Well, initially (sorry for the pun) I’m getting from the application configuration file (an XML file associated with the executable) the setting called CopyDirectories and setting the environment variable COBCPY to contain the setting value (old Micro Focus users will be familiar with this environment variable to set the path for finding COBOL copybooks). To set up application settings easily, right click on your COBOL project (or for that matter C# or VB), click on the Settings tab, give the setting a name, give it a type (in this case, System.String, but it could be an integer or a date for example), and finally give it a value:

SNAGHTML66d09f0

Visual Studio will then automatically generate an accessor class for you to easily read the setting, so you can just type:

Set settings to new type DirectoryUtility.Properties.Settings.

Set copyDir to settings::CopyDirectories.

to read the XML file.

I’m then in the Initialise method, off to the Windows Registry to the key  “HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Micro Focus\Visual COBOL\1.2\COBOL\Environment” (I’m running on a 64 bit version of Windows 7; if you’re running 32 bit you’d use “HKEY_LOCAL_MACHINE\SOFTWARE\Micro Focus\Visual COBOL\1.2\COBOL\Environment” … If I was smart I could even detect which version etc.). I then iterate through all the values within this key and set all the environment variables that Micro Focus requires to run the 32 bit compiler (like COBDIR and PATH).

We then have the Compile method that takes two strings, program name and directory. It then sets up a System.Diagnostics.ProcessStartInfo object with the parameters to pass to the compiler (cobol.exe), creates a System.Diagnostics.Process to execute the compiler, redirects output from the compiler to a stream, waits for the compilation to complete and reads the stream and displays the output.

Finally, to execute the functionality we have a little program:

Program-id CompilePrograms.MainProgram.

working-storage section.

01  Parms-Passed        Pic X(100).
01  dirCollection       type DirectoryUtility.DirectoryCollection.
01  startTime           type System.DateTime.
01  endTime             type System.DateTime.
01  duration            type System.TimeSpan.

procedure division.

    Accept Parms-Passed from Command-Line.
    
    If  type System.IO.Directory::Exists(Parms-Passed) 
        Set startTime to type System.DateTime::Now
        Display "Commencing compilation at " startTime
        Set dirCollection to new 
                    DirectoryUtility.DirectoryCollection(Parms-Passed)               
        Invoke dirCollection::Compile()
        Set endTime to type System.DateTime::Now               
        Display "Completed compilation at " endTime
        Display dirCollection::Count " programs were compiled"
        Set duration to endTime - startTime
        Display "Duration of compilation is " duration
        Display "Hit enter to continue"
        Accept Parms-Passed
    Else
        Display "Directory does not exist: " Parms-Passed
    End-If

    goback.

end program.

which reads the command line for the directory name, checks that the directory exists, takes a time stamp, creates the collection, compiles the cobol programs and takes the duration and prints the duration, ala:

CompileAllSequential

Note those stats: 46 minutes 18 seconds to compile 3044 programs. Not bad, eh? Let’s see what we can do next post to improve this!

The code can be found here.

Missing in combat

Sorry about the pause over the last couple of weeks … I visited Ireland for a week to work with a client and then had a week of everything going wrong. First the extremely hot weather broke my server badly, then it took a couple of days for a visit from the maintenance engineer to visit with replacement parts, only to find that other parts were needed and a pause for another couple of days. All the while NOT getting emails. Also, just prior to this period of electronic isolation finding my Visual COBOL license had expired I requested from a kindly soul a new license, but of course could not see it … All’s well now and I have my license so normal service is about to be resumed.

I’ve decided to briefly digress from the school project to create a little application that will compile every COBOL program in a directory. The aim of the program is to illustrate a fantastic feature of the .NET Framework: Collections and along the way show the Task Parallel Library in .NET 4.0 and spring delegates on you … I’ll do some coding today and have something for you soon

Fame?

I don’t know if you remember me mentioning last week I was filmed in preparation for the Visual COBOL 2010 R3 launch today. Well you can now see me in all my … errr… something! I realise that walking about a kilometre rather briskly in 80% Sydney humidity and being told to sit down and be filmed immediately might make me look a little damp … No, honestly, I’m not breaking out in a cold sweat and boy I wish I’d had time to squeeze in that haircut!  Here I am:

 

Check out the whole launch here

The meek shall inherit?

Sorry for the pause … Making a living can interfere with life at times.

Every COBOL programmer out there knows that the most important thing about Object Oriented Programming is inheritance, right?  To us old procedural programmer’s it just sounds so cool! I remember when I first started reading about inheritance in OOP I thought it was the answer to most of our programming problems.  Increasing complexity of existing code could be handled by simply inheriting code already written without breaking it. Turns out I was wrong … Err, mostly!

By and large it would be a GOOD IDEA if you didn’t design new classes that inherit the classes that you write. Note the qualifications there: by and large as well as that you write! There will be times that you might have good reason to inherit but, resist the urge if you can! You will build a far more flexible and far less complex system if you do. The other qualification was that you write … You will inherit .NET Framework classes all the time. That was what they were designed for, by experts. They provide functionality that it would be foolish to re-write. If you restrict yourself to this discipline you will reduce inheritance nesting to a minimum. Your code will be more performant, predictable and easier to debug. There is nothing more confusing than trying to work out where you are in an inheritance hierarchy that is over head height!

O.K. Now I’ve said what you shouldn’t do, what the hell do you do? You certainly don’t re-write the same functions everywhere. In the .NET world there are two basic methods:  use decomposition and/or interfaces. I will come to interface programming in .NET soon. Today is about decomposition.

Let’s start thinking on the basic school system we’re working on. It’s time to introduce Teachers. On first flush you might think a teacher is just a type of Person that we’ve already defined, so let’s just inherit from the Person class. Wait a minute though … What about the Teachers who are also Parents? What about the Teacher who also is Head of English or also does some school maintenance? Remember under .NET a class can only inherit from one class …

Well, first and foremost the Teacher is a Person who has other attributes or properties, like Position and a collection of Classes they teach (we’ll get to collections and a neat feature called Generics in the next post, but you’ll see a little of them in the code I’ve put together for this post). If we look at the Teacher class:

class-id School.Data.Teacher public.

working-storage section.

01  Teacher-Id       binary-long                        as "Id"             
                     public property with no set.
01  Teacher-Person   type School.Data.Person            as "Person"         
                     public property with no set.
01  Teacher-Position string                             as "Position"       
                     public property with no set.
01  Teacher-Classes  type School.Data.ClassCollection   as "ClassesTaught"  
                     public property with no set.

method-id New.
local-storage section.
procedure division using by value teacherId        as binary-long
                                  teacherPerson    as type 
                                      School.Data.Person
                                  teacherPosition  as string
                                  teacherClasses   as type 
                                      School.Data.ClassCollection.
                                  
    Set Teacher-Id          to teacherId.      
    Set Teacher-Person      to teacherPerson.  
    Set Teacher-Position    to teacherPosition.
    Set Teacher-Classes     to teacherClasses.                        

    goback.
end method.

method-id ToString public override.

procedure division returning returnString as string.

Set returnString to String::Concat(Teacher-Position ", " 
                 Teacher-Person::Title " " Teacher-Person::LastName).

end method.

end class.

You can see we have decomposed a Teacher into an Id, a Person, their Position and their ClassesTaught. Simple to write and simple to understand! We can enhance the underlying logic in the Person class without breaking the Teacher class and similarly do the same for the Teacher class. We can create Unit Tests for the Person and Teacher classes (using a technique called mocking) independent of each other and be confident in the functionality of each without worrying about dependencies.

If you are interested in some of the finer, more technical points of the argument of choosing composition over inheritance you might like to check out this article at StackOverflow (an excellent source of .NET wisdom) or perhaps this article at Wikipedia.

You can have a squizz at the code built so far here.

Horses

Given today’s my daughter’s 9th birthday and on school holidays today she’s getting a treat and (no self interest here) my son and I are taking her to see Tron Legacy at our local cinema. So sorry my planned posting will be appearing a little later as I have to get some work done beforehand … Instead, I thought you might all appreciate some more Eddie Izzard: