Roy Tang

Programmer, engineer, scientist, critic, gamer, dreamer, and kid-at-heart.

Blog Notes Photos Links Archives About

I’m being asked to assess whether we can safely upgrade the java version on one of our production-deployed webapps. The codebase is fairly large and we want to avoid having to regression test everything (no automated tests sadly), but we’ve already encountered at least one problem during some manual testing (XmlStringReader.getLocalName now throws an IllegalStateExeption when it just used to return null) and higher-ups are pretty nervous about the upgrade.

The current suggested approach is to do a source compare of the JDK sources for each version and assess those changes to see which ones might have impact, but it seems there’s a lot of changes to go through (and as mentioned the codebase is kinda large). Is it safe and easier to just review the java version changes for each version? Or is there an easier way to conduct this assessment?

Edit: I forgot to mention the version upgrade being considered is a minor version upgrade, i.e. 1.6.10 to 1.6.33

Posted by under notes at #java #questions #stackoverflow #software development
Also on: stackexchange / 7

Comments

sjr
sjr
Going through the JDK source changes sounds insane. How can you determine whether one source change is going to affect you or not? This sounds way harder than running up another instance of your app and testing it manually for regressions, no matter how big your app is.
EthanB
EthanB
Hopefully the incremental release notes will suffice.
jtahlborn
jtahlborn
Nothing is going to replace testing it in a real system. you may be able to catch something blatant in a bug report or in visual inspection, but detecting changes due to more complex interactions will be impossible. or even detecting seemingly simple changes which alter how the GC affects your running app or how hotspot optimizes your code (you are checking the c++ code too, right) or how some key algorithm performs…
sjr
sjr
The release notes won’t suffice.. They will say something like, “Class.getMethods now returns results in a different order to previous JDK”. How do you know for sure that your app will still work with this if you don’t test it out? Are you sure that you don’t depend on the old behavior? Bite the bullet. Test the app. :)
cringe
cringe
Wow, tough requirements for an update, especially a minor update. :-( Analysis of JDK sources sounds very theoretical to me and I don’t think you can say if there will be problems afterwards anyway. I’d probably start to think about building automated testssuites now, because this demonstrates the pain of untested systems. And after testing the main usecases on the current system you can setup a new test environment with the updated system.
Preet Sangha
Preet Sangha

As @jtahlborn says: Nothing will replace testing it properly.

I would go further and state that without automation then this is cost a you will occur again and again.

The correct answer is to

  1. Define a regression
  2. Run it (verify it)
  3. Automate as much as possible as you go through it

A simpler scenario is to simply run it and catch the errors are you or your customers find time. Personally I think this is good way to get demotivated developers, managers and customers. I strongly recommend you don’t use this approach.

Joseph Rajeev Motha
Joseph Rajeev Motha
This is where you start appreciating the effort you wouldve put into doing automated testing! :)
On This Site
Things I Write About
Elsewhere
Subscribe