jusblad

Making your own NetBeans modules

2011-10-01T22:02:00.024+10:00

NetBeans 7 is a powerful, free IDE that provides lots of opportunities to customise. If you find yourself continuing to create code that does roughly the same thing (eg. PHP scripts), you may want to make your own intelligent, dynamic template file to make life easier. The technology used to do all this is NetBeans IDE API and FreeMarker - a templating system that provides heaps of functionality, only a small portion of which needs to be used in NetBeans when creating modules.

As an exercise, I decided to make my own PHP template file module that lets you create a new PHP file that connects to a MySQL database and table, and runs a simple query against it. Because I keep forgetting the code (and because most of my PHP scripts do MySQL access), I decided to set up a NetBeans module that allows to you to creates a new PHP file with the shell code for database table access filled in. To use the templated file, it's just a simple right-click to get a panel where you provide a database name, a table name and the fields you want. Out pops a valid PHP script that opens the database, runs a SELECT * FROM xxx etc. query against it, then prints a list of fields you have selected in the panel.

There are some pretty good tutorials to do all this on the NetBeans sites, but a number of gotchas as you utilise version 7 - some of the tutorials are slightly out of date for this version. This posting should hopefully help you resolve those differences, and create your own customised template. Follow the tutorial on the NetBeans site NetBeans File Templates Tutorial while you read this post - I'm really just flagging a few tips to get it going on version 7, with a real-life example.

We'll be using FreeMarker as a template language to link what you put on your 'new file' panel into the actual PHP file you create... this is the timesaving feature.

You'll need the FreeMarker plug in for NetBeans, and I don't believe it's available from the standard plugin sites contained in Tools/Plugins, so you'll need to install the .nbm file manually.

Get the Freemarker Samples .nbm by using the link on NetBeans and FreeMarker Tutorial

You should read this page top-to-bottom to get an understanding of FreeMarker's interface to NetBeans, and can even try out the tutorial to install a standard Java file creator if you wish. If you just want to download and install the .nbm file, get it at NetBeans FreeMarker Sample plug-in

Press download, store it somewhere, then use Tools/Plugin/Downloaded and click 'install' to add it to NetBeans. It comes with a few samples that are immensely useful in understanding this beast.

Unfortunately, it won't compile out of the box unless you remove an unnecessary dependency to org.netbeans.modules.templates. Right click the project (FreeMarkerSupport) and edit the libraries list to remove it.

As the tutorial goes on to say, add in the other required (but missing) dependencies: namely 'Java Project Support', 'Project API', and 'Project UI API'. You should now have a working FreeMarker plug in with a few samples (HTML file, Java file, and even a little form-letter writer).

Now, to create your new module, select New Project, with a type of NetBeans Modules/Module. Give a meaningful name eg. PHPMySQLTemplate, and an appropriate code base name eg. org.yourorganisation.phpmysqltemplate.

Once you've done this, you can create the relevant files, compile and (most importantly), right click the project after compilation to Install/Reload in Development IDE. Create a wizard outline by following instruction 4 on the NetBeans FreeMarker Tutorial.

This will give you most of the files you need namely: Java panels, a java Wizard Iterator, a module description html, and layer.xml. You will be adding your own empty file to use a template (.ftl) file.

Because the purpose is to create our own panel, don't follow instruction 6 - we will customise our panel to collect a couple of fields and use those in the template. (Instruction 6 leaves out panel creation by just substituting the default Java file panels).

Now: on to your own panels. This was a trial-and-error thing for me, but the FreeMarkerSupport Example files were enormously helpful.

For your xxxVisualPanel1.java file, just add the required text boxes/list boxes etc. you need to provide the dynamic data for your eg. PHP file.
There's nothing too complex in here, but two things are important:
1) Add your template variables eg.

static final String DB_NAME = "db_name";
static final String TABLE_NAME = "table_name";
static final Object[] FIELD_NAMES = {"field_names"};

2) Provide accessor methods for your screen text boxes eg.


public String getDBName() {
return jTextField1.getText();
}

Note the Letter example shows you how to get hold of ListBox contents if you need that too.

For your xxxWizardPanel1.java file, you need to provide more accessor functions to get hold of the dynamic variables eg.


public String getDBName() {
return ((PHPMySQLVisualPanel1) component).getDBName();
}

Most importantly, don't forget to update the StoreSettings method with your dynamic screen variables eg.


public void storeSettings(Object settings) {
((WizardDescriptor) settings).putProperty(PHPMySQLVisualPanel1.DB_NAME, getDBName());
((WizardDescriptor) settings).putProperty(PHPMySQLVisualPanel1.TABLE_NAME, getTableName());
((WizardDescriptor) settings).putProperty(PHPMySQLVisualPanel1.FIELD_NAMES.toString(), getFieldNames());  
}

This step is the mechanism by which the dynamic screen variables get passed to the template file.

The xxxxWizardIterator.java file links it all together. In the instantiate method, you create a HashMap containing the dynamic variables off the panel. Follow the example file closely so you fill in everything. In particular, note the linkage done by these lines of code:


hashMap.put("db_name", wizard.getProperty(PHPMySQLVisualPanel1.DB_NAME));

This puts DB_NAME off the panel into the HashMap which will be provided to the FreeMarker engine to create a variable you can use in your template like ${db_name}. Sticking to the lower/upper case standards here is helpful ie. make your template variables lower case. All the other code outside instantiate stays as is.

The fun bit is creating your template. I used a PHP file, but of course your new file can be any type you like. The FreeMarker site has a list of all the macro instructions you can provide (eg. #ifs, and <#list). A useful site to explore FreeMarker syntax is at FreeMaker Template Cookbook. Again, follow the example .ftl files to help create a template that should run/compile properly as soon as you add it to your next project.

The layer.xml file is very important. Use an example and modify it from there. Important things here are:
folder xxx - tells NetBeans where the new file can be accessed eg. Templates/PHP
You must also add the following line:


<attr name="javax.script.ScriptEngine" stringvalue="freemarker"/\>

so that NetBeans knows you want to tell FreeMarker to use templating.
You can also make an .html file to describe your wizard file creation, and for completeness, you might want a 16x16 icon to show off your new file. There are two .html files in use: one describes the wizard itself, and the other, some information on the file you are creating.

Lastly, clean/compile and Install/Reload in Development IDE to install your template.

When you compile, be careful of some of the libraries you linked in. The main issue I had was the auto-import fixer brought in javax.tools.FileObject which is NOT the one you want: you want the wizard custom file object version:

import org.openide.filesystems.FileObject;

If all goes to plan, you can now add a new file to any project, filling in a couple of custom items on the panel, then see these correctly inserted at the right places in your new file. As I mentioned, I used all this to create a ready-to-go PHP file that accesses a database table and prints a few fields.

One last gotchya when it comes to using your new template: if you're using the ListBox, you'll need to actually select all the values you want: a list box full of data, but unhighlighted brings in sweet FA!

Given you are controlling the creation of your templated-file with FreeMarker and java code, you have opportunities to do all sorts of exciting things - my next idea is to actually pre-fill the table fields in the list box based on the selected database and table. This MySQL access code will go in xxxVisualPanel1.java file, once the user has chosen a database and table.

In summary, you can use NetBeans to 'intelligently' create new files for you using FreeMarker templating, and using Java code to interact with the NetBeans IDE. Nifty eh?

Sorting in C#

2010-12-23T20:08:00.008+11:00

Sometimes we all yearn for the days of DOS sort.exe, or MVS JCL's DFSORT. Sorts were big in the 2-3GL world of COBOL and the like as indexed files were not a given, and sequential file processing was the rule. Batch processing typically ran 'jobs' in steps, and one of those steps was inevitably a sort. Downstream process logic had precise dependencies on input data sort integrity and big-iron OS's all came with their own SORT utilities. The beauty of the sort step was it's clarity: you knew exactly what you wanted, and it was pretty easy to get it.

Although sorts have dropped from their prime position in the pantheon of programming processes, they're still around. Uncommon, but when required, utterly unavoidable.

Almost all programming languages make provision for sorting, and C#'s no exception.

C# sorting comes in a few flavours. The simplest is the sort method that pops up implemented in all sort of useful .NET library set objects like Lists. Of course, this sort of solution is rarely practical given the restriction of having only one sort operand, and we more often end up having to delve into the world of interfaces and OO approaches involving object derivation from useful .NET library components.

I recently had need to do a slightly-complicated sort on multiple keys, handling strings, numbers and datetimes. Here's a quick guide for this kind of situation, with a couple of learnt-the-hard way snippets.

1) Create a custom sort function.
This function is tasked with loading your data into a List (or some other aggregating .NET object type), defining and using your new custom sort IComparer object on this list and lastly, outputting your newly sorted List object into the output file. Running the sort itself involves calling the .Sort method on your list object, passing in the custom comparer object as the parameter.

2) Create a custom IComparer object
The magic (if you can call it that!) happens here.
IComparer-like objects simply compare two bits of data and return a 1 if the first is 'bigger', -1 if it's 'smaller' and 0 if they're the same. It's up to you to implement the logic behind bigger and smaller and in the process you can analyse the records to sort to process date times, strings and numbers (or even other objects). The comparison logic resides in the Compare method you create in this object - it's compulsory.

Two things are important to remember to get this object working correctly:

a) ALL comparison logic paths must be tested. The compiler picks up the fact there are missing paths for you if you forget some, but I guess there's only so much it can do. To abuse a consulting world term, your testing must be 'MECE' : mutually exclusive, completely exhaustive.

b) You should explicitly test for NULL against all the bits of the records you want to compare.

You can see that if you want to compare multiple parts of an object/record, you're going to end up with lots of if's and else's...especially given you need to check NULLs as well. The code can get quite ugly!

The result of your hard work implementing custom sort functionality is quite impressive speed-wise. A few hundred thousand record file with 3 comparison operands runs in 5 or so seconds on my average-fast PC. This is your reward for obeying the .NET paradigm - a somewhat verbose but very well-specified and slightly abstract piece of code works efficiently!

Here's a slightly schematic chunk of code explaining the above.


public static void SortMyFile() {
    // Read the whole file into a List
    List myLines = new List();
    using (StreamReader r = new StreamReader("the file.txt")) {
        string theLine;
        while ((theLine = r.ReadLine()) != null)
        {
            myLines.Add(line);
        }

    // Create the sort object and sort 
    MyVeryOwnComparer myVeryOwnComparer = new MyVeryOwnComparer();
    // Call the List.Sort method with the sorter as parameter.
    myLines.Sort(myVeryOwnComparer); 
    
    // write out your sorted list 
    // using StreamWriter or the like
           .
           .
}

// Your iComparer derived comparing object.
// Create the tests in the Compare method you must implement.
// Don't forget all paths to be tested and test for NULL explicitly.
public class MyVeryOwnComparer : IComparer
{
    public int Compare(string x, string y) 
    {
       // break up your x and y strings, 
       // convert elements as required and return
       // 1's, -1's or 0's
               .
               .
               .
    }
}

Java and C together

2009-08-31T15:57:00.021+10:00

It may seem antithetical, but your bloated java code can utilise your nice lean C libraries quite easily using the JNI interface.

There are two major tasks to get your C stuff into Java:

A. Modify and compile your C file using JNI hooks:

1) Make a special Java C header by calling javah on your .c file:

javah myOriginalCFile.c -o myJavaCFile.h

2) Make a new JNI C file out of your original C file and include the JNI interface and your just-made javah header file. Export your functions to Java by modifying the function signature. Use JNI's special types (eg. jdbouble) for anything coming into your function from Java:

#include <jni.h> /* Java Native Interface headers */
#include "myJavaCFile.h" /* Auto-generated header created by javah -jni */

JNIEXPORT jdouble JNICALL
myJavaCFunction(JNIEnv *env, jobject obj, jdouble x)
{
double x = 0.0;
/* Call the original C function */
y = myOriginalCFunction(x);
return y;
}

3) Compile your file into a shareable lib (architecure specific)

B. Use your new .lib in Java - there are 3 steps:

1) declare your C function in Java using the native keyword. 'native' says: it'll come from a library later. Do this in a static block of your class:

static {
private native double myCFunction(double x);
}

2) Get the library and load it. Note that you may want to set the library path either in the call arguments (-DLD_DYNAMIC_LIBRARY) or call set_library_path. Note that you don't need to include the file ending: Java will look for a .dll, .so, or .dylib depending on where you're creating your file.

System.loadLibrary("myLibName");
// or .dll or .dylib etc.
// System will check for right ending itself

3) Call it! Make a new object of your class, and use the function eg.

MyClass mc = new MyClass();
double myRtnValue = mc.myCFunction(2.33);

Tada.....C goodness for free!

For a more complete (and much better description!) of this process, see the java documentation from Sun at:
java.sun.com/docs/books/jni/html/jniTOC.html

For a link to a practical implementation of a bessel function:

www.nag.co.uk/doc/techrep/html/tr3_00/TRExample1.html

Bessel functions are solutions to Bessel's equation, and are useful for analysing wave propagation and decay in spherical or cylindrical coordinate systems: they are used extensively in signal processing and acoustics. Think drumskin vibrations varying over time. If you want to chart the y-displacement of the skin after your bang the drum, ask a Bessel function.

Why does Carbon form 4 covalent bonds instead of 2?

2009-08-28T11:16:00.005+10:00

Most people are not in the slightest bit interested in the answer to this question. For the 1% who are though, this vexing little question is at the heart of organic chemistry - that bitch of subject that seems to drive people alternately mad/ecstatic. I always knew C formed 4 bonds, and got through uni organic chem and some other stuff without really knowing/remembering why. So....years later, I decided to sort myself out.

Here's the answer:

Carbon's valence shell is configured 2s2 2p2 (remember the p shell can contain up to 6 electrons, in a 3-D 2px, 2py, 2pz shape). Two half-filled p orbitals should mean stable molecules like CH2 - the 2 electrons from the H filling the 2px and 2py subshells. But....these don't normally exist ...instead carbon likes to form 4 covalent bonds with other atoms. Why 4 and not 2? Because the 2s2 2p2 shell hybridizes. After hybridization there are NO LONGER any s or p shells. This is really important to understand. They've gone forever!

2s2 2p2 becomes : sp3 sp3 sp3 sp3 (the hybrid).

What's an sp3 shell look like? sp3 shells look nothing like s or p shells...more like a very short stocky baseball bats. They're called sp3 because the s and the 3 p shells combine to make 4 sp3 orbitals. Because there are 4 of them, and they like to be as far away from each other as possible, C forms (single bonding) tetrahedral shapes ie. 4 triangle surface planes.

I'll try to put some picture in to explain this better...I admit that wasn't a great job. The most important thing I realised was that sp3 orbitals are not s and not p. They are completely different, and the old s and p orbitals have gone.

How does a double bond work? Similar deal: let's take ethylene as an example:

H2C=CH2

1 x s and 2 p orbitals create 3 x sp2 orbitals. The remaining p orbital on the 2 C atoms forms a pi bond.

A triple bond's no different: 1 x s and 1 x p form an sp orbital. 2 p's then form 2 pi bonds. Between the carbon atoms, you've got one sigma bond (from the sp's touching), and 2 pi bonds to make 3! (the sp hybrid has nothing to do with the normal p pi bonding). Easy eh?

Hopefully this explanation also clears up that confusing issue of naming the hybrid orbitals. sp3 for 4 bonds, sp2 for double, and sp for triple never made any sense to me, until I realised they're just the formula that represents how many of the old s and p bonds combined.

Where are our paths?

2009-07-12T19:08:00.003+10:00

Lost when it comes to Leopard's paths? They're all important when you compile your own stuff and will drive you nuts at some stage, so after quite a bit of stuffing around (my incompetence, not theirs), I've decided to list the path to path in ORDER.

The key thing to note is that in Leopard, in addition to the standard /etc/paths, Apple have created a 'path_helper' script (called from /etc/paths), that picks up path directories from two new special locations.

1) A text file call /etc/paths
2) any file you put in /etc/paths.d

Before this, your own .bash_profile is read from ~/.bash_profile or, for root, /var/root/.profile in the standard way.

So...the order for your paths is:
~/bash_login or ~/.profile (if you have you path like xxxxx:$PATH)
/etc/profile (mine contains JAVA_HOME)
/etc/paths
/etc/paths.d/*

If you've installed your own software in /usr/local, you'll probably want to edit /etc/paths first, as this gets set up first too, and you generally want to override /usr/bin, /bin, /sbin and /usr/sbin to get your own stuff going first.

The general commands in your script are simply
XXXX=/your/path
export XXXX

Entropy PHP and xdebug on Leopard

2009-07-12T17:36:00.009+10:00

Like everything else *nix, Leopard comes with a crippled PHP. It's a PHP nonetheless, but as soon as you want to write interesting things you'll find it doesn't have many important modules compiled in.

I mucked around with compiling my own into /usr/local/mysql but found the missing link was always GD....a bugger to compile in. Not wanting to use the bloated, and often outdated ports or fink versions, I relented and finally used a pre-compiled binary from entropy.ch provided by Marc Liyanage. This dude in Switzerland maintains a great binary, Leopard package installable version of PHP with lots of goodies. It's at
http://www.entropy.ch/home, and it sure beats stuffing around with endless compile options from the raw php code.

Of course, one other goody you'll want is xdebug by Derick Rethens - another deutschesprechende by the looks of it. This fantastic addition lets you debug your PHP scripts realtime, and hooked up to Komodo, Netbeans or Eclipse for instance, allows you to lazily trace through your code, inspecting variables (and changing them) as you go. It's a godsend to PHP developers everywhere and very popular, with new features added all the time. You can find xdebug at
http://xdebug.org

To get xdebug going in Marc's binaries you need to take care of a couple of gotchyas.
1) compile it with CFLAGS=-arch x86_64 as that's what Marc's is.
>CFLAGS=-arch x86_64
>export CFLAGS

2) make sure your path finds Marc's PHP first, not Apple's or any other PHP's you've got lying around. This is really important, as otherwise you'll enter version issue in php.ini reminiscent of DLL hell! You do the standard phpize, and ./configure --enable-xdebug --with-php-config=path to php-config.

Manually move the modules/xdebug.so into Marc's directory (mine was /usr/local/php5/lib/extensions/20090623).
The last gotchya was tricky. You load your extra modules in the entropy version php in the php.d directory - just add a new file eg. 50-extensions-xdebug.ini. It will get picked up, however MOST IMPORTANTLY, even though you might have set your extensions_dir in the main php.ini, the [Zend] section in the 50-extensions-xdebug.ini file requires the FULL path to xdebug.so. Nothing else will do.

Thanks to both these guys for making Leopard PHP development easy. Vielen Danke (or, as they say in der Schweize Danke vielmal) dufte Typen weil Ihr PHP coding leicht gemacht habt! (thanks guys you make PHP coding easy).

Ruby - getting the latest on Leopard

2009-07-07T00:28:00.003+10:00

Wanna upgrade Leopard's default ruby binary? Unlike most of the other Apple inclGNUsions like MySQL and PHP, they couldn't really cripple this one. Still, we all like the latest greatest stuff right?

Per usual, the process can be painful. To cut a long story short, with a new 64-bit MySQL and PHP installation, you need to do this (in addition to your other ruby upgrade steps) to get the MySQL part going.

env ARCHFLAGS="-arch x86_64" gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config

Otherwise - you'll bomb out as Leopard tries to compile a universal binary ie. a whole lot of binary stuff you don't want/need and can't use. The use of the --with-mysql-config flag is a nifty way to get these things to just WORK with your MySQL installation. Kudos to MySQL for this.

DBI and DBD::mysql on Leopard

2009-06-13T16:22:00.002+10:00

I needed (wanted?) to use Perl's DBD::mysql module last night. OK, I thought, a quick perl -MCPAN -eshell and install should do the trick, as Leopard's Perl usually plays quite nicely with CPAN. Damn.....error after error in the compilation (the C bit), so off to google for a fix. OMG WTF AFAIK a quick google on using Perl's DBD::mysql module in Leopard brings up a whole pile of pleading postings from desperate and frustrated people. Cutting out the clueless postings (the ones with page after page of compilation errors and pointless 'try this' - 'try that' replies), it seems the problem is nearly always the same thing: the mysql socket, or rather, it's location. Two factors conspire against you using DBD::mysql - Apple's surprise exclusion of the MySQL headers, and DBD::mysql's expectation (I think) of a hard coded location for the mysql.sock file.

To sort this mess out
1) Get Apple's 'we're sorry here's the MySQL stuff' and install it. It's at http://support.apple.com/kb/HT3370
2) Fix your php.ini file to use the same socket as DBD::mysql. I guess you could do it the other way round, but the Perl .pm module adjustment looked harder. Don't know where to find your php.ini? Shame on you! Anyway....
>php -i|grep php.ini will tell you where it lives
>nano your_location/php.ini and change mysql.default.socket = whatever DBD::mysql wants. In my case, it was /var/mysql/mysql.sock.

OK, so now everybody should be using the one socket, so
>apachectl -restart
and everything should be working. Check php -i|grep socket to make sure.

home automation

2009-05-16T02:27:00.005+10:00

Bunnings is a day-spa for fellas. Satisfying our desire to make or fix things - real things, not sit-on-your-arse-in-an-office things, the place lets you think you're saving money too through DIY (although this is a fallacy - see below). The stores are cold and industrial looking, giving them instant cred in a world where every other shop is decked out like a funky bar and you feel pretty tough as you stride the gigantic aisles with shelves holding every kind of widget imaginable, almost literally like a kid in a lolly shop. But while the massive range of stuff is probably Bunning's greatest asset, not far behind would be the staff and their attitude to customers. Where else does your standard-issue office guy get a chance to have a hardware-related conversation with (an apparently) real tradie? Certainly not with the ones who come to your house who seem to love confusing you with jargon, and even take delight in shaking their heads as they tell you the problem is much, much bigger than you thought.
Nope, the Bunning's guys are almost always friendly and helpful, and happily talk to you as if you're one of them as you choose your 1/4" self-tapping holster bolts. For the 80 odd percent of us guys out there who earn money by doing stuff with Microsoft Office, a trip to Bunnings is a psychically nourishing experience.

* You don't save money at Bunnings: you end up going there 5 times to get 'just one more thing' for your pet project, and you end up with a whole lot of unused leftovers that are the result of impulse buying before you know what you actually would need if you had a clue. Some examples from my recent trips include:

Useless	Useful
A bucket of tile grouting	Cans of WD-40
Half-used cans of Liquid Nails	Wrenches
Dozens of super-glue tubes - dried hard	Nails of varying sizes

Leopard and iconv

2009-05-12T22:32:00.005+10:00

Anyone who uses Leopard's PHP realises pretty quickly that the binary is lacking some very basic modules, not the least of which is MySQL. Why provide MySQL, PHP, and a *nix platform and not put MySQL as a module into the PHP object? Don't try to answer this question - there is no answer. Put it down to Apple cuteness I guess.

If you're PHPing on a Mac, your also probably recompiling your PHP object regularly, to pick up modules your latest pet needs. Chief among these as mentioned before would be MySQL, but gettext, curl, crypto and SSL would be up there too.
Compiling these monster binaries on a Mac is not quite the *nix goodness one would hope for given the Darwin foundation - google around and you'll find a top 3 or so problems list that if nothing else, shows you that other people round the world are tearing their hair out too!

My biggest problem always seems to be iconv. It seems Apple's version does/does not (?) include the functions lib_iconvopen and libiconv_close (or at least the .h files don't redefine the functions). Simple you might think: get a new iconv that works, compile it into /usr/local and away you go. Wrong. You'll probably get a scrambled message telling you iconv functions ain't found or don't work.

I always forget how I get out of this quagmire, so here I am documenting it here so I won't lose it, and you might find it helpful!

::::::: AT THE LAST COMPILATION STEP WHERE libphp5.so IS CREATED, MANUALLY ORDER YOUR -L/libs and I/includes ARGUMENTS SO THE COMPILER PICKS UP CONSISTENT HEADER AND LIBRARIES GIVEN YOUR NEW MODULE BINARIES::::

By the time the PHP make gets to this step, there are duplicated -l's -L's and -I's all over the place. Unfortunately, -L/usr/lib can come up first - making all the work you did compiling new add in modules useless. Simple eh? Yet it's amazing how many postings on various fora are out there still wondering how to deal with this nasty, but basic little problem. So...don't rely 100% on the output of ./configure - some handcrafting will get you to the Khubla Khan of 64-bit PHP your way.

Commercial + java. Huh?

2009-04-27T01:13:00.034+10:00

Being a semi-interpreted language (I'm not trying to start a religious war here), copy protection of java based software seems pretty lame at the moment (forever??). Yes, there are obfuscators out there, but at the end of the day, the .class files have to be processed by a JVM, leaving little to hide.

Now people have lots of reasons for decompiling, not all of them nefarious and in my case recently, I just wanted to see what the hell the code was actually doing, as well as make a note of any open source libraries used. Being provided with a Mac OS X app bundle, you first need to explore the contents (right click - Show Package Contents and the java files should be in Resources somewhere. Don't forget a .jar is just a .zip). Once you've found the goodies, you can start the decompilation process. My needs for a decompiling this particular app were simple: process a large number of class files in batch, keeping the directory structure intact, so that I could then use NetBeans to debug it as a new project.

There are number of tools out there to decompile Java, and a great one that has a neat GUI interface is Monsieur Emmanuel Dupuy's straightforwardly named 'JD-GUI'. While it produces nice looking code, deals with Java 6, and has a good interface, it only works on one .jar or .class file at a time. It's written in C++ using wxWidgets, which accounts for its speed. One great feature is that you can now take a whole .jar and save all its sources in a zip file with one click. Unfortunately, the author has no plans at this stage to release the code, or, maybe more usefully, 'JD-Core', the C++ library that does all the hard work, so for big java decompilation jobs, you'll have to go elsewhere unless you can be bothered decompiling each .jar individually. You can get JD-GUI at http://java.decompiler.free.fr/

The venerable JAD decompiler by Pavel Kouznetsov has been around forever, but is strangely now very difficult to find. The original domain kpdus.com has gone, but Tomas Varaneckas has kindly tracked it all down and is hosting all the versions on his own site at http://www.varaneckas.com/jad.

Let the decompilation begin...

OK....armed with JAD, I began the task. A good process is a repeatable process, so here's a quick guide for anyone inclined to muck around with their own java decompilation.

1) Copy your compiled java directories somewhere new - jad can break stuff if you're not careful

2) Run jad using a few special options to recursively get all the .class files, rename the default suffix of the decompiled code from .jad to .java, and put the source code somewhere OUTSIDE the .class directories. Here's a bash script I wrote to do all this quickly:

#!/bin/bash

# use jad Java decompiler recursively on all 3 class dirs,
# putting .java files into ./src
# options are:
# -o overwrite without confirmation BE CAREFUL IN PRODUCTION DIR
# -r recursive .classs search
# -s append .java instead of .jad. BE CAREFULE IN PRODUCTION DIR
# -d the directory you want decompiled INTO eg. src
# Last parm: the files you want to decompile
# com/**/*.class = in dir com, ALL child directories, all .class files.
~/Desktop/jad158g/jad -o -r -sjava -dsrc com/**/*.class
~/Desktop/jad158g/jad -o -r -sjava -dsrc nmfl/**/*.class
~/Desktop/jad158g/jad -o -r -sjava -dsrc org/**/*.class
echo Finished decompiling com, nmfl, org.

The ** operator is interesting: it tells jad to recursively go as many layers deep as it need to pickup all the class file.
While the script runs, you'll probably get a few message saying jad couldn't completely interpret some things eg. try { catch { stuff, and some specific IO things in particular. I'm not sure if this is Java 6 specific or not.

Once you've run this script, go into the ./src directory and look at your nice new .java files. You'll get a new little header comment telling you Jad decompiled this work. Ignore the link - kpdus.com is no more.

Decompilation is great, but if the code's been obfuscated, you've still got a bit more work on your hand. I'm no expert at the obfuscation side of things, so I'll put my project into NetBeans, have some guesses at the bizarre class names (d4eJH etc). and maybe do some refactoring if I can work out what's going on.....

Stay tuned!

not another blog...

2009-04-26T23:49:00.005+10:00

Q. Why did you create a blog?
A: My virtual stuff is growing in size, and I need a place to put it.

Not being able to remember stuff I've learnt about nerdy stuff annoys me, and I'm getting so much of this stuff now that I'm in danger of my virtual world stresses spilling over into the real world. The other downer about virtual stuff overload is that the more you get, the harder it is to find the one bit you really, really need, right now.

That's one reason. This is a second:

You know that feeling you get when you're stuck trying to work out a solution to a problem that's got you completely stumped, google it up with a few choice keywords and bingo, find an answer?

mmmm.....feels good.....

It feels so good because a) you know the writer went through the same crap you're going through right now, b) they worked out a solution and c) they bothered to write it down on their blog so others could benefit.

If I can bring that feeling to a few people I'll be really glad I ~~succumbed~~ ~~got sucked-into~~ joined the blogosphere!