So I’ve finally got my act together and opened a twitter account.
For anyone interested, I’m http://twitter.com/TimCreswick (@TimCreswick)
Container-managed authentication is very useful in Java web applications; it saves writing a lot of back-end authentication and authorisation code into our applications, makes SSO (single sign-on) trivial between J2EE apps and also allows developers to hook into existing enterprise authorisation frameworks with relative ease.
With our recent development work with the Yubikey, a one-time password (OTP) token from Yubico we decided that it would be useful to have a Yubikey authentication realm for the Glassfish application server.
A lot of our existing J2EE applications use the JDBC Realm that is included in the Glassfish distribution. The JDBC Realm allows you to configure the authentication of your application to any JDBC back-end accessible by your application server (i.e. enabled by an appropriate JDBC driver library). Typically this will take the form of a MySQL database table with a list of usernames and password hashes.
Here’s an example of the Glassfish v2.1 JDBC Realm configuration template:
Glassfish v2.1 JDBC Realm Configuration
Our custom Yubikey Realm is essentially an extension of the JDBC Realm (although not in the sense of a Java “extends”). The Yubikey Realm supports all the same functionality as the JDBC Realm, so you can drop it in-place with all the same settings and it’s totally backwards compatible.
We’ve added the functionality to specify an additional column in the users database table containing the Base 64 Yubikey Public Identifier (ID) of the Yubikey assigned to that user. This is configured via the yubikey-column property. By default, the Yubikey Realm will authenticate users as normal, unless the password length is >= 32 characters. In this instance, the password is assumed to be a Yubikey OTP, and the Realm authenticates accordingly.
We have the additional parameters yubikey-force and yubikey-auth-url, which respectively control whether all logins are treated as Yubikey OTPs, and which validation server you would like to use.
Here’s an example of the realm configuration screen:
Yubikey Authentication Realm Configuration
…and there we have it. Instant in-place implementation of Yubikey authenticated logins for all of our existing J2EE applications without changing any application code. Obviously it’s worth noting that in this context we’re using the Yubikey OTP as a convenient password replacement, not as a 2nd authentication factor.
Please feel free to contact me if you’re interested in using our authentication realm; we may consider publishing it online in due course, along with the installation instructions and a few enhancements that are pending.
I’ve recently been evaulating Two-factor Authentication (2F) implementations for one of our clients, including the excellent Yuibikey product from Yubico.
The initial requirement for our client project is 2F protection of all remote-access to their systems, which presently only amounts to VPN connections and webmail access via Outlook Web Access (OWA) provided by Exchange 2007.
We’ve now finished implementing this, and the diagram below shows how we protected the OWA service (I might cover VPN in a later post). Naturally there are a wide variety of ways to provide 2F authentication for OWA, but I think this presents the best end-user experience:
Authentication Process Overview
First, a quick overview of the highlights:
Examining our final implementation step by step:
Hopefully I have provided a coherent overview of our solution. I will try and follow this up with more details on the custom ISA forms, the LDAP schema modifications and FreeRADIUS integration in future posts.
As always, feedback is welcome!
I mentioned in this post (relating to my Solaris ZFS / iSCSI management script) to the storage-discuss OpenSolaris mailing list that we were mostly using NFS as opposed to iSCSI for our ESXi backing stores, and was asked by Christoph Jahn to provide some background on this.
Here’s my response as posted to the list:
Hi Christoph.
Several reasons, I’ll try and outline those that I can recall:
1. We found the performance of NFS to be far better than iSCSI, although we probably hadn’t spent sufficient time tweaking the iSCSI configuration. NB: we were exporting ZFS block devices as LUNs to virtual machines as RDMs (raw device mappings) as opposed to formatting them as VMFS, so the comparison isn’t perhaps completely fair.
2. Overall the administration of iSCSI was overcomplicated, and prone to human error. Training our people to administer and troubleshoot it was likely to be too costly, and the additional ongoing costs associated with the increased administration have to be considered.
3. Similar to point (2) above, we found that you had to be careful to keep track of IQNs and LUN Ids when mapping in VMware, it’s only possible to put names on targets, not LUNs, so when mapping multiple disks into a VM the process was potentially error-prone.
4. The provisioning of iSCSI storage between ESXi and COMSTAR *felt* a little bit unstable; we had numerous instances of lengthy HBA rescans, targets appearing and disappearing which although they were all explained within ‘expected behaviour’ were either a little counter-intuitive or too time-consuming.
5. My understanding of the nature of ESX NFS connections is that less IO blocking takes place, which explains why so many people get better throughput when running multiple VMs against the same SAN connection.
6. The limit of 48 NFS mappings in ESX wasn’t going to be a constraint for us for the foreseeable future; we rarely load more than 15 VMs per node. The flexibility of creating a separate ZFS filesystem for each VM and exporting it separately over NFS was therefore possible.
7. Everyone understands files; when you login to the Solaris system you can navigate to the ZFS filesystem and see .vmx and .vmdk files – these are nice and simple to manage, clone, backup, export etc etc.
8. Related to point (7), when you use NFS in this way the virtual machine configuration is stored alongside the rest of the virtual machine data. This means that a snapshot of the ZFS filesystem for that VM gives us a true complete backup at that point in time.
I think that just about covers it. Essentially, although iSCSI feels like a more clever way of solving our problem, and certainly it poses as a more ‘enterprise’ solution, it really was a case of overcomplicating the solution.
Several times in the last few years we’ve found that simple is better, even if it doesn’t satisfy one’s desire to implement the ‘technically perfect solution’. It’s more a question of balancing the economics (I’m running a business, after all) with the actual requirements. Oftentimes just because you *can* do something doesn’t mean you should, and I’ve often found that PERCEIVED requirements can out-grow ACTUAL requirements just because some technology exists to solve problems that you don’t have [yet].
In short, I like to keep it fit for purpose, even if it does feel like a more agricultural solution.
Finally, I should add that the one major shortcoming with our NFS solution is the lack of any equivalent to the iSCSI multipathing. If we had any machines that required true high availability or automated failover this would probably have negated all of the points above – iSCSI multipathing is a beautiful thing, and it creates some awesome possibilities for fault tolerance.
As it stands, we take care of link failure at the network level (as opposed to the iSCSI MPIO protocol level) and deal with ESX node or storage node failure by manually remapping NFS filesystems from elsewhere. This is actually preferable to automated recovery since sometimes we don’t want to take the ‘default’ action during a failure scenario. By having a carefully documented failure plan I believe we have more flexibility, and can deal with recovery on a per-client basis, rather than a system-wide basis.
Ultimately we are a business dealing with multiple clients hosted on shared hardware, so it’s important to keep our implementations client-centric, rather than system-centric.
Finally, I should add that although we don’t have automated failover of these systems, our solution does still permit us to stay well within our contracted SLAs, which serves the business need.
Regards,
Timothy Creswick
My primary reason for buying an eBook Reader was for the large number of technical PDFs, white papers and carefully formatted user guides that I read for work.
I found it very hard to find how well these pre-formatted PDFs would display on the device, with a lot of people saying that they were barely legible. Obviously with most of these documents you don’t want to spend time reformatting them for the Reader’s screen, and zooming is cumbersome and often breaks the formatting.
Other people’s posts notwithstanding I went ahead and bought one.
The good news is that I find it perfectly usable, and although the rendered font size ends up being rather smaller than you might choose, text and diagrams are rendered very clearly. In case it helps anyone else deciding to buy one of these devices for this kind of work, I’ve included some photos below.
First, a regular eBook page rendered in “Small” (click for hi-res):
Then a page from a Sun technical manual (click for hi-res):
Finally, here’s a close up including some very small text, all of which I find perfectly fine for my purposes. Again, click for a larger version, and apologies for the lens distortion which I haven’t bothered to correct:
The default PowerShell prompt is long and irritating. I switch my default to the following:
A better PowerShell prompt
In order to achieve this, I have the following script:
function prompt { Write-Host("%") -nonewline -foregroundcolor Yellow; return " "; }
c:
cd Scripts
" ";
You’ll also note that I change the default working directory to C:\Scripts, which is where I keep all my PowerShell scripts and saves me a little extra time.
In order to implement this, you’ll need to create the folder C:\Users\{USERNAME}\Documents\WindowsPowerShell and save the file as profile.ps1.
We’re using one of our Solaris machines to publish iSCSI targets to some VMware ESXi hosts (initiators) for client virtual machines.
To make provisioning new LUNs as simple as possible I’ve written an interactive script that first creates a ZFS backing store, then creates the LU for STMF, gives you the option of creating a new iSCSI target or using an existing one, and finally attaches the LU to the selected target.
Here’s a sample output from the script:
----------------------------------------------------------------- ZFS / iSCSI / COMSTAR Simple Management Toolset (c)Copyright 2009 Thoughtspace Ltd. Developed by Timothy Creswick ----------------------------------------------------------------- Please specify the name for the new ZFS backing store. Your new store will be created inside mpool/clients/iscsi Existing backing stores are: NAME USED AVAIL REFER MOUNTPOINT mpool/clients/iscsi 271G 973G 19K /mpool/clients/iscsi mpool/clients/iscsi/testing 24K 973G 24K - mpool/clients/iscsi/testing-three 24K 973G 24K - mpool/clients/iscsi/testing-two 24K 973G 24K - New backing store: mpool/clients/iscsi/test Should the backing store be created as a sparse volume? [Yes/No] no Please enter the size of the backing store: 10GB Will create a ZFS store with the following details: Name: mpool/clients/iscsi/test Size: 10GB Sparse: No Proceed? [Yes/No] y Creating ZFS backing store... OK Creating the STMF LU... OK (GUID is 600144f094ab000000004aa29327000b) You can either attach this LU / device to an existing iSCSI target as a new LUN, or you can create a new iSCSI target and STMF target group for this ZFS backing store. In general, you want to have a single iSCSI target per Virtual Machine, so if the virtual machine requires multiple disks, then you should attach each additional disk as a new LUN on an existing target. If this is the first disk that you are attaching via iSCSI to the Virtual Machine you should create a new iSCSI / STMF target. Create a new iSCSI target? [Yes/No] y Creating iSCSI target... OK (iqn.1986-03.com.sun:02:2fff96a8-63b2-48bd-b447-bda2a1fc9ead) Creating STMF target group... OK Assigning new iSCSI target to target group: Waiting for STMF service to terminate.... OK Adding TG member... OK Starting STMF service.... OK Adding LU view to target group... OK All done. Your ZFS device should be available on target iqn.1986-03.com.sun:02:2fff96a8-63b2-48bd-b447-bda2a1fc9ead.
You can get a copy of the script here. Please feel free to use and modify – I’d be interested in any feedback.
Nmap is a great tool; utterly indispensable for network administrators. If you haven’t got it yet goto http://nmap.org or install via your favourite package manager.
Here’s one of my frequently used nmap commands – very simple, but gives you a quick list of IP addresses for all of the ProCurve switches on your network.
nmap -n -sP 10.0.0.0/24 | grep --before-context=1 "ProCurve Networking by HP" | grep "Host" | cut -f2 -d' '
Just change the subnet range to match your network. Note: this relies on doing vendor OUI lookups on device MAC addresses, which means that the scanning node needs to be on the same Layer 2 network segment for this to work.
I recently took delivery of a new Dell Latitude E6400 laptop to use as primary workstation. All things considered it’s turning out to be an excellent choice.
One of the things that surprised me was the presence of a GPS transciever built into the laptop as part of the HSPA 3G wireless card. Viewing in Windows Device Manager it appears as a component connected to the USB sub-interface on the card:
Device Manager Screenshot
The interface to the GPS unit is simple NMEA over the virtual COM port 7. By default there’s not much you can do with this – Google Earth will interface pretty much out of the box, but not much else.
I decided it would be fun to write a little Java application to map the current position of the laptop, as given by the GPS unit.
I looked around a bit and found JXMapKit, which is part of the swingx project. Using some of this tutorial to get me started, I had a basic mapping application up and running in about 20 minutes.
The next step was to interface with the GPS unit. This is where Java falls down a little, espcially on Windows machines. In the end I settled for the RXTX library which uses native implementation (JNI) to provide access to serial devices from within a Java app.
The Latitude’s GPS unit spews out location information on the COM port as it becomes available, so all you need to do is ‘listen’, and then handle the stream in an event-driven manner.
A sample GGA ‘fix’ sentence looks like this:
$GPGGA,123519,4807.038,N,01131.000,E,1,08,0.9,545.4,M,46.9,M,,*47
I couldn’t find any NMEA parsing libraries for Java online, so another 2 hours and I’d written my own. There’s a core class which validates the NMEA sentence and then invokes a specific handler class depending on the NMEA sentence type. For example, GGA (fix) sentences are handled by NmeaLocationMessage, GSA (overall satellite data) sentences are handled by NmeaSatelliteGeneralDataMessage, GSV (detailed satellite data) sentences are handled by NmeaSatelliteDetailMessage and so on.
Each of these ‘handler’ classes contains the logic to further parse the specific sentence type. Just to get up and running I only implemented handlers for GGA, GSA, RMC and GSV messages, which were more than enough to get a fix on the map.
Finally I wrote the code to invoke the NMEA parser on serial port events. The core event handling code looks like this:
NmeaMessage message = new NmeaParser().parseSentence(line);
if (NmeaLocationMessage.class.isInstance(message)) {
NmeaLocationMessage location = (NmeaLocationMessage) message;
if (location.getLatitude() != lastLat || location.getLongitude() != lastLon) {
lastLat = location.getLatitude();
lastLon = location.getLongitude();
map.setAddressLocation(
new GeoPosition(
location.getLatitude(),
location.getLongitude()));
}
}
Nice and simple!
The net result is that everytime the GPS reports a new Latitude or Longitude, the map is re-centered accordingly.
So finally, here’s a location plot as I’m sitting on a train at London Paddington Station waiting to depart:
GPS Tracker Screenshot
I wrote this really for a bit of fun, since clearly it’s not massively practical. If you have an E6400 with GPS, or a similar notebook and would like to play around with it please let me know and I’ll gladly send you the app and/or sourcecode.
Tuning JVM Memory Settings
Update: Andrew Wang’s blog post (http://www.umbrant.com/blog/2012/twitter_jvm_tuning.html) is a pretty nice quick guide, well worth reading in conjunction with this page.
Sorting out sensible JVM memory settings in a production environment can be a tricky business. There are some great articles, blogs and lengthy white-papers out there explaining the details of how the JVM manages memory, but I’ve yet to come across a simple guide that actually helps configure the key JVM startup parameters.
For the impatient: Right down at the bottom of this page I have included a simple tool for helping to set your JVM parameters.
For those who are new to this, I suggest doing some web searches around “java heap”. Other people have done a much better job of explaining the theory behind this.
Whilst the theory is well worth reading, I’m hoping this provides some more practically useful information for people running production Java applications. In this context, I’m dealing with a Tomcat 7 server running on the Oracle JVM with an IO intensive application that performs a lot of in-memory caching.
First up, fire up VisualVM (included with your JDK in the /bin folder) if you haven’t already. This will allow you to interrogate the memory of your running application. If you’re trying to interrogate a remote system, you’ll want to run jstatd in order to get a connection. There’s information elsewhere on the web about how to do this; I’ll probably summarise it in another post later.
That will give you a beautiful screen like this:
The VisualVM Visual GC View
The server in this example has a 6GB heap allocation and 150MB PermGen allocation. The applicable JVM startup parameters are:
To run through these quickly; ‘server’ tells the JVM to run in server mode. This mainly changes the behaviour of the HotSpot compiler. In general things will take a little longer to ‘warm up’, but long-running performance will be improved. There are other implications too, which are well documented elsewhere.
Setting the Xmx and Xms values to be the same, in this instance 6GB, we create a single (hopefully un-fragmented) heap allocation for the virtual machine. It’s this 6GB that then get’s carved up into the areas shown in the screenshot above.
The PermGen space is assigned with MaxPermSize and PermSize. In general you don’t need to allocate much here; this mostly relates to the number of classes that you have loaded; relatively simple applications without loads of library inclusions get away with very small values here. We’re using 150MB in this example, which is more than enough. Again, setting the values the same saves the JVM scaling up and down with usage.
Now for the interesting stuff. The heap is split into Eden Space, Survivor Space and Old Generation. Without going into excruciating detail, objects are created in the Eden space and then moved to the Survivor Space and finally Old Generation. The idea is that the majority of the objects that you create only need to last for a very short period of time (consider an Iterator for example; it only needs to exist for the duration of a single loop). By creating objects in the Eden space, the initial garbage collections on this space are very efficient, since they potentially scrub very large percentages of all objects. Anything which is still in use is promoted to one of the survivor spaces, and then eventually to the Old Generation.
Performing garbage collection on the Old Generation is usually more time consuming, so that’s what you ideally want to avoid when scaling the other spaces.
It’s worth noting that the Survivor Space is actually split into two parts, known as S0 and S1. Only one of these is active at a time, and it receives the objects being tenured from the Eden space.
The JVM settings that you use to define these spaces are the ‘NewRatio’ and the ‘SurvivorRatio’.
The NewRatio is the ratio between the Old Generation and the sum of Survivor Space (both S0 and S1) and Eden Space (in this context, ‘New’ means Survivor and Eden). For example, a NewRatio of 2 will create an Old Generation that is twice the size of Eden plus Survivor (or 2/3 of the heap).
The SurvivorRatio is the ratio of one of the Survivor Spaces and the Eden Space. For example, a Survivor Ratio of 10 will mean that the Eden space is 10 times the size of each of S0 and S1.
So clearly it’s not a simple task to visualise how the heap will be allocated when you’re setting these values.
To that end, I have created this little bit of javascript which you can use to plug in your Xmx (heap size), NewRatio and SurvivorRatio. The resultant heap split is then shown in the table below. All byte values are in MB. Don’t forget to multiple GB values by 1024 to get MB.
I have already put in values from my example above; that is a heap of 6GB with a NewRatio of 2 and a SurvivorRatio of 10. This is actually an unusually high survivor ratio, however having watched our application in production for a few weeks we’ve been able to tune it to our [rather unusual] workload.
Input Values
Resultant Allocations
Please leave a comment if you have any questions – I’d be really interested to hear if this helps to solve your problems.