$createNestedParamStruct is a cfwheels badass

Written by jbriccetti on . Posted in Misc Ramblings

One of the nice things about the cfwheels framework is the ability to build forms that map directly to a model object using form helpers. On the “receiving” end (in your controller), you have access to a key in the params scope that is a nested structure of data that’ll line right up with your model. This stuff makes scaffolding CRUD forms a serious layup.

#select(label="org", objectName="contact", property="org_id",options=orgs, valueField="org_id", textField="org_name",includeBlank="Select an Org")#

Recently, I have been working on a standard set of scaffolding templates to take the CRUD generation to a whole new level. Along with a few methods in the base controller (and some helper methods for the view) I’ve worked in generic code to include pagination, column header sorting and filtering in all the generated templates. While I was working on the filter controls, I realized I’d seriously like to have a nested param structure with all the filter fields contained within. But the form helpers are specifically for mapping fields to a model object, and I’m not doing that. So I looked further into how wheels constructs those nested params for the form helpers and low-and-behold, i found a simple little function that when used with the right naming convention, tears it up on the form-field parsing front. And, it’s called on every request automagically, so you aint gotta do nuthin to make it work. Schvinnnng!

Looking inside /wheels/dispatch/request.cfm i found the $createNestedParamStruct() method. After discovering how it expects and parses field names, i then reverted back to my filter form and started naming fields like this:

#selectTag(label="type",name="filters[label_id]", options=labels,textfield="label_description",valueField="label_id")#
#selectTag(label="org",name="filters[org_id]", options=orgs,textfield="org_name",valueField="org_id")#
#selectTag(label="contact",name="filters[contact_id]", options=contacts,textfield="full_name",valueField="contact_id")#

Notice I’m not using the objectName or property arguments to the selectTag function (like we did in the select(0 function for the model object mapping) – selectTag doesn’t tie the form field to a model “object” But, by using the [] notation on the field name, wheels automatically creates a sub-structure in the params scope when the post occurs… so by the time I can even get my hands on these any of these params in the controller, it’s already neatly encapsulated in a sub-structure named “params.filters”

dump of params.filters

How to restore from svndump (or svn export)

Written by Perry Woodin on . Posted in Misc Ramblings

If you have ever had to move svn from one server to another, you were probably given a svndump file that contains all revision history. If you are like me, you favor a GUI over command-line. Unfortunately, most GUIs don’t support a load or restore.

So for ease of future reference, here is how to restore from a svndump file (sometimes called an svn export).

First step. Fire up terminal.

Note: You can find all of this info in the book Version Control with Subversion at http://svnbook.red-bean.com/

Get the svndump

You’ll probably get a svndump file from your svn host. If you already have the file you can skip this section.

If you need to create a svndump file you can do so with svnadmin dump. If you type the following into terminal, you’ll create the file repo.svndump. This file will contain a dump of your entire repository history.

$ svnadmin dump /path/to/repo > repo.svndump

Example:
$svnadmin dump /www/repos/helloworld > helloworld.svndump

Create a new empty repository

In order to restore the repository that was just dumped, you need to create a new empty repository. Here you’ve got a couple of options. Most svn GUI clients allow you to create a new repository. You can either go that route, or create the new repository via the command-line like so.

$ svnadmin create /path/to/new/repository

Example:
$ svnadmin create /www/helloworld

Load (restore) the svndump

Once you’ve got a new empty repository, you can load the svndump file. Here’s where most svn GUI clients let you down. In most cases, your only option is command-line.

$ svnadmin load /path/to/new/repository < /path/to/svndump

Example:
$ svnadmin load /www/helloworld < /temp/helloworld.svndump

Let that run until you see the command prompt again, and your new repository will be fully loaded.

Sync with a remote repository

If you need to restore to a new remote repository, you will need to use svnsync. I have only had success with this if the remote repository is new (empty). Here is the command-line sync.

$ svnsync init --username YOUR_REMOTE_REPO_USERNAME REMOTE_URL SOURCE
$ svnsync sync --username YOUR_REMOTE_REPO_USERNAME REMOTE_URL

Example:
$ svnsync init --username troyweb http://mysvnhost.com/helloworld file:///www/helloworld
$ svnsync sync --username troyweb http://mysvnhost.com/helloworld

cfsetting in cfscript

Written by jbriccetti on . Posted in App Dev

Obsessed with converting much (but not all) of my CFML to <cfscript />, I recently discovered there is no <cfscript /> equivalent for the <cfsetting /> tag.  Of course, there is always a way…

CFML

<cfsetting enablecfoutputonly="true" />
<cfsetting requesttimeout="60" />
<cfsetting showdebugoutput="false" />

<cfscript />

<cfscript>
    createObject( "java", "coldfusion.tagext.lang.SettingTag" ).setEnablecfoutputonly(true);
    createObject( "java", "coldfusion.tagext.lang.SettingTag" ).setRequestTimeout(javaCast( "double", 60 ));
    createObject( "java", "coldfusion.tagext.lang.SettingTag" ).setShowdebugoutput(false);
</cfscript>

 

SVN obstructed? How to remove all .svn folders

Written by Perry Woodin on . Posted in Misc Ramblings

I recently committed a rather large code base to SVN. To my knowledge, the code base had not been in SVN before. Turns out I was wrong. Upon attempting to commit the working copy, I kept seeing directories flagged as “obstructed.” I tried an SVN clean, but that didn’t help.

The problem was a bunch of legacy .svn folders in various directories. My fix was to fire up terminal and delete all of the .svn folders in one fell swoop. Here’s how to do it.

  • Start Terminal.
  • Change directory to your code base. This is really important. Otherwise you could unintentionally delete .svn folders you would rather keep.
    $ cd /Library/WebServer/targetdirectory
  • I like to list, just to make sure I’m in the right place.
    $ ls
  • Find and delete the .svn files
    $ find ./ -name ".svn" | xargs rm -Rf

Done. Now you should be able to commit without issue.

Preventing Browsers from Caching Resources with Spring 3…Sort of…

Written by Sean Ryan on . Posted in Misc Ramblings

Developing Web apps can be very frustrating sometimes since there are so many different browsers and environments that you need to consider just to get simple stuff working at times.  Not too long ago I had to tell a client to hold down the control key and hit F5 to force her browser to fetch a new file from the server instead of from her browser’s cache.  She wasn’t seeing the UI changes I made, but she was seeing the content. It looked horrible. That’s terribly embarrassing to say to a client and entirely preventable.

With Spring, you can use a WebContentInterceptor to modify properties of the request to help govern how a browser will handle resources that don’t change very often. Here’s a snippet from a config file that I’ve used:

<mvc:interceptors>
 <bean id="webContentInterceptor" class="org.springframework.web.servlet.mvc.WebContentInterceptor">
 <property name="cacheSeconds" value="0"/>
 <property name="useExpiresHeader" value="true"/>
 <property name="useCacheControlHeader" value="true"/>
 <property name="useCacheControlNoStore" value="true"/>
 </bean>
</mvc:interceptors>

Fairly straight forward. I’m saying to browser, don’t cache stuff.  In an actual production system, you’ll want to be a little more specific about what you want cached and not cached. In development though, it’s nice to never cache.

The problem with this is the browser can still do whatever it wants. So how do we control the browser? We don’t. Instead, we spoof the location of the resource and trick the browser into thinking it’s a brand new file – obviously not cached. This is the recommended technique and Spring handles it quite nicely. Check it out.

We use a resource mapping.

<mvc:resources location="/static,classpath:/META-INF/" mapping="/static-2.1.0/**"/>

This configuration says, for all requests that come in that have the URL pattern /staticapp-version followed by anything, look for the resource in folder named static in the web root or in a jar on the classpath within its META-INF directory.

Now, a web browser that requests a resource called /static-2.1.0/css/app.css will serve the app.css file from the same old location but since the browser doesn’t know that, it must download the app.css file regardless of whether or not a previous version was cached.

Cool.

We can improve upon this by making the version dynamic so you don’t need to modify your Spring config file each time you deploy.

<mvc:resources location="/static,classpath:/META-INF/" mapping="/static-${application.version}/**"/>

You can add a variable to a properties file and read that file into your Spring config file. Then you’ll be able to use the variable in a SpEL expression.

<util:properties id='appProperties' location='classpath:default.properties'/>
<context:property-placeholder properties-ref="appProperties"/>

Your default.properties file might look like this.

application.version=2.1.0

Just update this file on each deploy and all your static resources will be forced to be loaded from browsers.

You have much more control over what resources should be cached and the expiration of those caches than I’ve talked about here. For more information, I encourage you to explore the Spring documentation which talks about this very technique along with several other cache control strategies.

http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/mvc.html#mvc-interceptors

 

Switching Annotation Preprocessors for JPA Meta Model Generation in Eclipse

Written by Sean Ryan on . Posted in Misc Ramblings

In case you haven’t noticed, Eclipse doesn’t have the most stable annotation preprocessor. I use Hibernate as my persistance provider but I’ve experienced problems with Hibernate’s meta model generation jar – at least when used in Eclipse. In need of a solution, a colleague turned me onto OpenJPA for the meta model generation and still use Hibernate as the actual persistance provider. Open JPA is a bit too strict as a provider still but works like a charm for metamodel generation.

To set the annotation processor in Eclipse, follow the steps below. I’ll set it to OpenJPA, but you can use any one you want.

  1. Right click the project in the project explorer and go to Properties…
  2. In the left side navigation area, select Annotation Processing under Java Compiler.
  3. You’ll want to check Enable Project Specific Settings, then Enable annotation processing
  4. Select the generated source directory – I build with Maven, so I’ve selected the standard Maven location.
  5. You can see from the the screenshot that I have a single processor option openjpa.metamodel=true. You’ll need to set this if you’re planning to use OpenJPA for your metamodel generation. Hibernate does not require this. See the documentation of your processor if using something other than OpenJPA or Hibernate.

    Annotation Processor Screenshot - Spring Source Toolkit

    Annotation Processor Screenshot – Spring Source Toolkit

  6. Next, expand Annotation Processing in the left side navigation and select Factory Path.
  7. Here is where you’ll point to the jar file containing your metamodel generator. You can see that I unchecked the eclipse version and later tried the Hibernate version and now I’m using the OpenJPA version.

    Annotation Processing Screenshot - Factory Path

    Annotation Processing Screenshot – Factory Path

  8. Click OK and you should be good to go.

Also, you’ll want to make sure you update your pom.xml to use the same provider.

 

 

Relocating localStorage in iOS 5.1 for PhoneGap Applications

Written by Sean Ryan on . Posted in Misc Ramblings

See bug details here: https://issues.apache.org/jira/browse/CB-330

The general problem is that as of iOS update 5.1 (3/2012), Apple no longer considers data stored in localStorage or the SQL Lite DB to be persistent data. This means that any data stored using these HTML5 features is considered volatile across updates and process termination. To say the least, this is a problem for any developers replying on this data being available across app updates, backups, or crashes. It’s bad enough that we only get 5MB of storage (2.5MB if you’re storing 16bit text characters), now we can’t even persist the data when the app is run natively? That’s yucky.

Product groups like Sencha, Ext4.js, and  PohoneGap are working on work arounds for this new Apple specification but as of this entry, they aren’t all there yet. Until a fix exits for Sencha, I’ve taken the approach of changing the default location of localStorage and the SQL DB that resides within the app container on the device. This solution solves the problem entirely and has the added benefit of tons more storage capacity. The down side is that it requires modification of the native code-base and therefore requires apps for the iOS system to be built in xcode. Consequently, mobile app dev for iOS using this approach now requires a Mac. Lucky me, I have me some Macs.

The Fix

Copy and paste the following code into AppDelegate.m‘s method didFinishLaunchingWithOptions. The placement likely doesn’t matter but I add it as the first block of code.

You can read through the code to see that it won’t affect you older iOS devices and that it just relates where the localStorage mechanism point to on the device.

/* Fix problem with ios 5.0.1+ and Webkit databases described at the following urls:
     *   https://issues.apache.org/jira/browse/CB-347
     *   https://issues.apache.org/jira/browse/CB-330
     * My strategy is to move any existing database from default paths
     * to Documents/ and then changing app preferences accordingly
     */

    NSString* library = [NSSearchPathForDirectoriesInDomains(NSLibraryDirectory, NSUserDomainMask, YES)objectAtIndex:0];
    NSString* documents = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];

    NSString *localStorageSubdir = (IsAtLeastiOSVersion(@"5.1")) ? @"Caches" : @"WebKit/LocalStorage";
    NSString *localStoragePath = [library stringByAppendingPathComponent:localStorageSubdir];
    NSString *localStorageDb = [localStoragePath stringByAppendingPathComponent:@"file__0.localstorage"];

    NSString *WebSQLSubdir = (IsAtLeastiOSVersion(@"5.1")) ? @"Caches" : @"WebKit/Databases";
    NSString *WebSQLPath = [library stringByAppendingPathComponent:WebSQLSubdir];
    NSString *WebSQLIndex = [WebSQLPath stringByAppendingPathComponent:@"Databases.db"];
    NSString *WebSQLDb = [WebSQLPath stringByAppendingPathComponent:@"file__0"];

    NSString *ourLocalStoragePath = [documents stringByAppendingPathComponent:@"LocalStorage"];;
    //NSString *ourLocalStorageDb = [documents stringByAppendingPathComponent:@"file__0.localstorage"];
    NSString *ourLocalStorageDb = [ourLocalStoragePath stringByAppendingPathComponent:@"file__0.localstorage"];

    NSString *ourWebSQLPath = [documents stringByAppendingPathComponent:@"Databases"];
    NSString *ourWebSQLIndex = [ourWebSQLPath stringByAppendingPathComponent:@"Databases.db"];
    NSString *ourWebSQLDb = [ourWebSQLPath stringByAppendingPathComponent:@"file__0"];

    NSFileManager* fileManager = [NSFileManager defaultManager];

    BOOL copy;
    NSError *err = nil; 
    copy = [fileManager fileExistsAtPath:localStorageDb] && ![fileManager fileExistsAtPath:ourLocalStorageDb];
    if (copy) {
        [fileManager createDirectoryAtPath:ourLocalStoragePath withIntermediateDirectories:YES attributes:nil error:&err];
        [fileManager copyItemAtPath:localStorageDb toPath:ourLocalStorageDb error:&err];
        if (err == nil)
            [fileManager removeItemAtPath:localStorageDb error:&err];
    }

    err = nil;
    copy = [fileManager fileExistsAtPath:WebSQLPath] && ![fileManager fileExistsAtPath:ourWebSQLPath];
    if (copy) {
        [fileManager createDirectoryAtPath:ourWebSQLPath withIntermediateDirectories:YES attributes:nil error:&err];
        [fileManager copyItemAtPath:WebSQLIndex toPath:ourWebSQLIndex error:&err];
        [fileManager copyItemAtPath:WebSQLDb toPath:ourWebSQLDb error:&err];
        if (err == nil)
            [fileManager removeItemAtPath:WebSQLPath error:&err];
    }

    NSUserDefaults* appPreferences = [NSUserDefaults standardUserDefaults];
    NSBundle* mainBundle = [NSBundle mainBundle];

    NSString *bundlePath = [[mainBundle bundlePath] stringByDeletingLastPathComponent];
    NSString *bundleIdentifier = [[mainBundle infoDictionary] objectForKey:@"CFBundleIdentifier"];
    NSString* libraryPreferences = @"Library/Preferences";

    NSString* appPlistPath = [[bundlePath stringByAppendingPathComponent:libraryPreferences]    stringByAppendingPathComponent:[NSString stringWithFormat:@"%@.plist", bundleIdentifier]];
    NSMutableDictionary* appPlistDict = [NSMutableDictionary dictionaryWithContentsOfFile:appPlistPath];

    BOOL dirty = NO;

    NSString *value;
    NSString *key = @"WebKitLocalStorageDatabasePathPreferenceKey";
    value = [appPlistDict objectForKey: key];
    if (![value isEqual:ourLocalStoragePath]) {
        [appPlistDict setValue:ourLocalStoragePath forKey:key];
        dirty = YES;
    }

    key = @"WebDatabaseDirectory";
    value = [appPlistDict objectForKey: key];
    if (![value isEqual:ourWebSQLPath]) {
        [appPlistDict setValue:ourWebSQLPath forKey:key];
        dirty = YES;
    }

    if (dirty) 
    {
        BOOL ok = [appPlistDict writeToFile:appPlistPath atomically:YES];
        NSLog(@"Fix applied for database locations?: %@", ok? @"YES":@"NO");
        [appPreferences synchronize];
    }
    /* END Fix problem with ios 5.0.1+ and Webkit databases */

Next, copy and paste the following code in AppDelegate.h. Place it as the last line of code above @end

/* 
 * Returns YES if it is at least version specified as NSString(X)
 */
#define IsAtLeastiOSVersion(X) ([[[UIDevice currentDevice] systemVersion] compare:X options:NSNumericSearch] != NSOrderedAscending)

 

Writing Strongly Typed NOT IN Subqueries with JPA CriteriaBuilder

Written by Sean Ryan on . Posted in Misc Ramblings

This one took a while to figure out and it required a few beers.

Let’s say you have an entity called Product and an entity called ProductOwner and there is a 1 to n relationship from ProductOwner to Product but you can only access Product from ProductOwner.

This may not be the way the database is designed – that would be terrible, but it may be the way your model has been designed. Probably an oversight but in any case, now you’re stuck with it.

What do you do if you want find out how many ProductOwners don’t actually own any products? It can happen, but because of the design of the model, it’s not obvious how to fetch those records.

In SQL, it’s easy…

select * from ProductOwner po where po.id not in (select p.id from product)

Using JPA CriteriaBuilder you would do the following.

CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<ProductOwner> query = cb.createQuery(ProductOwner.class);
Root<ProductOwner> poRoot = query.from(ProductOwner.class);
query.select(poRoot);

Subquery<Product> subquery = query.subquery(Product.class);
Root<Product> subRoot = subquery.from(Product.class);
subquery.select(subRoot);

Predicate p = cb.equal(subRoot.get(Product_.productOwner),poRoot);
subquery.where(p);
query.where(cb.not(cb.exists(subquery))); 

TypedQuery<ProductOwner> typedQuery = entityManager.createQuery(query);

List<ProductOwner> result = typedQuery.getResultList();

Dealing with Case Sensitivity of Database Table Names at the Persistance Layer with Hibernate

Written by Sean Ryan on . Posted in App Dev, Architecture, Gotchas, Strategy

It’s been a while since my last post so it’s about time I got back to it since I’m backlogged with all sort of awesomeness.

Recently, I took over a pretty big Spring project. Lucky for me, it was written largely in Spring 3.0 so my periodic upgrades to 3.1 haven’t been too difficult. I’m the only developer on the project but it’s not that bad (anymore) and it’s given me a chance to really focus on my Spring skills.

Since I develop locally, I backed up the production database on the server and brought it down to my laptop. Before I could work on it, I needed to run the data through a conversion routine that migrates the database model and its data into the new and improved model for the next release.

The Problem

The problem is that the server is Linux and my computer is a Mac.

Why does this matter? It matters because each OS has different case sensitivity. The conversion routine uses Hibernate as the persistance provider and it’s configured to use the DefaultComponentSafeNamingStrategy which maintains the same case as the managed entities. This is fine except that the application uses a custom naming strategy that converts everything to lower case and acts as a wrapper around the DefaultComponentSafeNamingStrategy. The only time this would ever cause a problem is when the operating system the application is run on is case sensitive. Linux is but Windows is not case sensitive and neither is Mac. I didn’t know this. In fact, I was under the impression Mac was a good OS.

Any of my collegues reading this right now are smiling because I’m constantly listing the ways Windows is terrible and Macs are better as programming tools.

The Effect

This problem wasn’t actually a problem on Windows or Mac. It manifested itself when I moved my converted data up to the server and restored it into the database. When I fired up the application, all the data was gone! Pretty sure a pink slip was in my future, I scrambled to figure out what happened. This was the first time the data was updated since my firm took over the project and they had put their faith in my ability to not blow the database away. I had just blown the database away, or so I thought. It took a while to figure out what happened.

As it turns out, the data was there but with camel casing but since the app uses a lowercase naming strategy, it wasn’t able to find any of the tables.

For example, a table named MyWidgetTable was being looked for under the name mywidgettable.

Finding the Best Solution

This might seem like an easy fix, just update the naming strategy for the conversion routine. It’s true, that’s the best solution and the one I selected but, it’s important to realize why it’s the best solution.

For instance, another option would be to configure the database – in this case, MySQL – to use lowercase naming when creating objects. People do this. You can read more about that here http://dev.mysql.com/doc/refman/5.0/en/identifier-case-sensitivity.html.

Although this is an option, I don’t recommend it.  Doing so will more tightly couple the database implementation to the application and whenever possible we like to make each tier in an architecture as pluggable and removable as possible.  Changing databases has more side effects when you start introducing configurations that directly impact the design of a system.

In general, when you need to deal with filesystem objects, try to choose an implementation that will work across the big players to keep your code as portable as possible even when you think you’re environment won’t change. It will and in this field, crap changes every couple of days! The previous developer knew this and that’s probably why he wrote the custom lower case strategy to begin with.

More Fun with ColdFusion & Spreadsheets

Written by jbriccetti on . Posted in App Dev

I do like spreadsheets. They’re basically glorified csv files and talk about the lowest-common-denominator for data – I’m pretty sure after the nuclear holocaust, the last two things left on earth will be cockroaches and data stored in spreadsheets…

There are many ways to work with spreadsheets, but it’s not uncommon to be pulling data into ColdFusion queries, massaging the data, and then stashing it. This is the sort of utility script work that I get asked to do on client sites like it’s my job. Oh, wait… yeah yeah…

Like most data structures, ColdFusion queries are great for some jobs and stink for others. But then there are things about queries that should work well but they simply don’t. If you have ever tried to generically output a query and you used the “queryname.columnlist” property, you know what I’m talking about. That column list is in alphabetic order, not in the order it appeared in a select statement, or in this case, in the order of the spreadsheet columns. Oh, and its translated to upper case, which is seriously ugly. This is not always a problem, however, when working with spreadsheets, it often is. Spreadsheet users usually put their columns in an order for a reason – and if you are working with a script to read a spreadsheet into a query, do a little dance and then write a new spreadsheet back out, generically (i.e. without hard-coding the column names in your script) – then using that columnlist property for the headers in the output spreadsheet is trouble.

Solution? use the getmetadata function on your query to get an array of structs that describe the columns in the query – and yes, that’s an array, which means ordered, in this case from left-to-right in the spreadsheet. Nice! Also, the case of the values is completely preserved. Extra Nice!

One trick is that you now have an array of structs and what you usually want (for your output to the spreadsheet) is just a common-delimited list of the column names. The column names are all in the key “name” in each structure item in the array. Oh, look what i found (and by found i mean wrote)  http://www.cflib.org/udf/arrayofStructsToList

So, drop that <cffunction /> into a page and sprinkle in some of this sample code:  note, you’ll also need a source xls file

<cfscript>
    fpath = expandPath(".");
    sep = request.vars.sep;
    source = listappend(fpath,"source.xlsx",sep);
    dest = listappend(fpath,"dest.xls",sep);
</cfscript>
<!--- pull the source spreadsheet into a query object --->
<cfspreadsheet action="read" src="#source#" query="q" headerrow="1" rows="2-9999" />

<cfoutput>ugly stuff:#q.columnlist#</cfoutput><br />
<cfoutput>much better:#arrayofStuctsToList(getmetadata(q),'name')#</cfoutput><br />

<!--- image you wanted to modify the data and do some calculations --->
<cfset queryAddColumn(q,"check",[]) />
<cfloop query="q">
    <cfscript>
      isok = "BAD";
        // do whatever you need to do here
        if (true) isok = "OK";
        querySetCell(q,"check",isok,currentrow);
  </cfscript>
</cfloop>

<!--- note the newly created column comes back in both calls. thats good! --->
<cfoutput>ugly stuff:#q.columnlist#</cfoutput><br />
<cfoutput>much better:#arrayofStuctsToList(getmetadata(q),'name')#</cfoutput><br />

<!--- create an output spreadsheet for the new data  --->
<cfset out = spreadsheetNew()>
<!--- Add header --->
<cfset spreadsheetAddRow(out, arrayofStuctsToList(getmetadata(q),"name"))>
<!--- format header if you like --->
<cfset spreadsheetFormatRow(out,{bold=true},1)>
<!--- drop in the data --->
<cfset spreadsheetAddRows(out, q)>
<cfset spreadsheetWrite(out, dest, true)>