My Love/Hate Relationship with WordPress

Written by Perry Woodin on . Posted in Misc Ramblings, Software

WordPress is really amazing. I can standup a WordPress site on Amazon’s EC2 in under 15 minutes. And we’ve got a developer here at Troy Web who is adept at writing plugins. So, I love WordPress because it can facilitate launching a basic website in under a day.

What I hate about WordPress is it is quite often the wrong tool for the job. That’s not really the fault of WordPress. People become enamored by its ease of use and try to shoehorn functionality that is probably better served by custom application development. At a certain point they feel locked into the platform. That’s never a good thing.

But what really prompted this post/vent is the ubiquity of WordPress plugins. I recently helped a friend move from WordPress.com to a self hosted install. We had to replace some widgets and it took me hours to find a suitable plugin. Not because the plugin didn’t exist but rather because there were hundreds of plugins that claimed to offer the same functionality. My new rule for plugins is only install if the number of downloads exceeds 250,000.

So, if you want a WordPress site, I’m happy to help, but lets talk about your site’s goals before you commit to the platform.

User Input, Part 1 – Encoding

Written by jbriccetti on . Posted in App Dev, Gotchas

The Problem

Security 101 – ensure you are implementing input validation to prevent XSS. From the OWASP Top 10 Attack Vector #2

You need to ensure that all user supplied input sent back to the browser is verified to be safe (via input validation), and that user input is properly escaped before it is included in the output page. Proper output encoding ensures that such input is always treated as text in the browser, rather than active content that might get executed.

Example 1: outputting request parameters

<cfset form.name = "Jon" />
Thank You, <cfoutput>#form.name#</cfoutput>

In the above example we are setting the form field “Jon” But if we collect the value from a user (like forms normally do), AND if  the user supplied the name as follows:

<cfset form.name = "<script>document.location='http://en.wikipedia.org/wiki/Cross-site_scripting'</script>" />
Thank You, <cfoutput>#form.name#</cfoutput>

What happens when the output gets “displayed” to the screen? What happens is an instruction is executed by the browser! The script instruction redirects the browser to wikipedia. Imagine if that website was Evil. Really Evil. We’re not talking “the diet coke of evil” or the “margarine” of evil, we’re talking Real Evil. Bad news for your browser.

So we have to figure out a way to tell the browser not to treat these characters as instructions? But before we try to figure out how to do that, what characters are we talking about?

 

character encoding
 <  &lt;
 >  &gt;
 “  &quot;
 ‘  & #39;
 &  &amp;

The bracket characters you already know are characters that can embed a <script> tag into a page. These are big naughty characters. Big. Naughty.

But what about the ” and the ‘ character? Well, checkout this example:

<cfset form.name = 'Jon" onMouseOver="javascript:alert(document.location);"' />
<input name="name" type="text" value="<cfoutput>#form.name#</cfoutput>" />

So it’s pretty obvious where this is going. Any characters that are used for markup should be encoded, lest they be used for big naughty things. Or small naughty things for that matter.

The solution (or at least a solution)

The simplest of solutions is encoding the data on the way out (when sent to the browser). So in the above examples, if we were to simply squeeze our output through the built in ColdFusion function xmlFormat(), we’re good to go:

<cfset form.name = "<script>document.location='http://en.wikipedia.org/wiki/Cross-site_scripting'</script>" />
Thank You, <cfoutput>#xmlformat(form.name)#</cfoutput>

<br />
<cfset form.name = 'Jon" onMouseOver="javascript:alert(document.location);"' />
<input name="name" type="text" value="<cfoutput>#xmlformat(form.name)#</cfoutput>" size="100" />

 

Next Steps:

In part duex, we’ll look at a slight drawback to using the xmlFormat() function. Also, we’ll examine an approach to encode the input on the way in. Finally, we’ll look at filtering input to wipe out any other characters that are used for naughty purposes, such as tabs, carriage returns and other non printable characters. Until then, stay safe…

 

 

A Mobile App with Offline Maps using Titanium

Written by Tim Varney on . Posted in App Dev, Mobile

This past week I was presented with a typical request, but with an unusual twist.  I needed to create a Titanium mobile app which features custom raster and vector layers.  The challenge: the map had to be available when the end user was not connected to the Internet.

Titanium’s implementation of the Ti.Map object doesn’t support offline maps, so I knew right away that I’d have to find a third party solution.  I’m not an Objective C or Java programmer, so writing a custom module was out of the question.  That left me one option.  I needed to find a JavaScript mapping library which I could present in a webview.

During my research, I came across a blog post that mirrored what I wanted to do.  Scott Sheri had created an offline map mobile app using PhoneGap, Leaflet and TileMill which was very similar to what I wanted to accomplish.  Seeing how Scott had blazed the trail for me, I decided to follow his path, only using Titanium instead of PhoneGap.  I knew there would be a lot of differences in implementation, but it gave me a strong starting point.

Generating the Raster Images with TileMill

Years ago, I spent a lot of time working with GIS data, so I have a very strong understanding of how to produce and display spatial data.  I chose TileMill to produce my raster layer because it was easy use and readily saved the images in the mbtiles format.  The beauty of this is that an .mbtiles file is nothing more than a SQLite database.  I could imbed my generated .mbtiles in my application and query it natively using Titanium.

Pay no attention to my horrible choice of colors.  I won’t go into details about how I produced this map.  TileMill has some pretty good tutorials on their website.  Needless to say, what we’re concerned about here is how it’s formatted when it’s exported.

The mbtiles files produced by TileMill are simple SQLite databases.  The schema can be found on the MapBox website.  To view the data within the file, I’m using SQLite Manager add-on for Firefox.

Now that we have our raster layer, we can work on getting Leaflet to work in a Titanium webview.

Using Leaflet in a Titanium Webview

This was relatively simple.  I created an HTML page which included all of the JavaScript needed to produce a Leaflet map.  I then took the two Leaflet source files (leaflet.js and leaflet.css) and included them in the Titanium /Resources/ directory to allow it to run offline.  I then created a Titanium webview to display the page.  Quite quickly I had Leaflet up and running in Titanium.

Extending Leaflet to Utilize a SQLite .mbtiles Database

Leaflet doesn’t support .mbtiles out of the box.  The good news is that because Leaflet is open-source and well documented, it was possible to extend its functionality.  I extended L.TileLayer to create a new JavaScript constructor for adding mbtile layers.  The new constructor is L.TileLayer.MBTiles.

L.TileLayer was intended to add raster layers where the images are server by a tile server with a standard URL format.  (For example, http://{s}.tile.cloudmade.com/[API-key]/997/256/{z}/{x}/{y}.png.)  It was going to be necessary to modify this so that the images would instead be drawn from a BLOB field in the .mbtiles SQLite database.

This is where my implementation had to differ from Scott Sheri’s.  Because Scott was using PhoneGap, he queried the .mbtiles database using a SQLite plugin.  With Titanium, I have native database functionality and would be using that instead.  The big hurtle was that the main Titanium app and the webview are in different JavaScript contexts and I would have to pass information between the two using the Ti.App.addEventListener() and Ti.App.fireEvent() methods.  The fireEvent() method is not asynchronous and does not allow for a callback function.

I overrode three of the L.TileLayer methods (_loadTile(), _createTile(), and _addTilesFromCenterOut()) and added one new method (_onLoad).

  • _createTile() – Originally, the Leaflet tiles (which are simply img html elements) didn’t have an id attribute that I could reference.  This was going to be a necessity because the fireEvent() and addEventListener() methods are not asynchronous.  I was going to have to reference the tile by id when the Titanium response for a request for a tile was returned.  I simply created a random guid and set it as the id.
  • _loadTile – I removed the setting of the tile’s src attribute in this method.  I placed the relevant information needed for requesting tiles (i.e. x, y, z, and id) in an array named aUpdate.  When loaded, the map will consist of blank tiles and they will be populated by Titanium through a later request.
  • _addTilesFromCenterOut() – Leaflet uses a document fragment to load the img tiles to the DOM.  I had to wait until this document fragment was added before I could send the request to Titanium to query for the actual image data.  At the end of this method, I just added a call to _onLoad().
  • _onLoad() – This new method will fire once the blank map tiles are added to the DOM.  It calls a Titanium application level event named getMbTiles.  It passes a JSON form of the aUpdates array.

The getMbTiles event triggers the query of the database for the needed tiles.

SELECT tile_data
FROM images
INNER JOIN map ON images.tile_id = map.tile_id
WHERE zoom_level = ?
AND tile_column = ?
AND tile_row = ?

These tiles are then base64 encoded and sent back to the webview using a custom event named receiveTileUrl.  receiveTileUrl sets the src of the various tiles to the appropriate image data.

Source Files

Leaflet Demo  (does not include mbtiles file) .zip

MasterView.js

map.html

The Offline Map!

Ta-da!  The images are now shown on the Leaflet map imbedded in a Titanium webview.  By turning on airplane mode will show that all the data and code is hosted locally and that no Internet connection is required to use this map.

Next Steps

Now, this is just a proof-of-concept and is not production ready.  The current obstacle is the size of the .mbtiles file.  My simple demonstration map was about 4mb and contained very little visual data.  A more robust raster layer can easily exceed the 50mb limit imposed by the Apple App Store.  A strategy needs to be developed where the end user can pick and choose what data they will need in the field and download it accordingly through the app.

But that’s another blog entry…

Migrating fckeditor 2.6.4 to ColdFusion 10

Written by Perry Woodin on . Posted in App Dev, Gotchas

I moved a site with FCKEditor from ColdFusion 8 to ColdFusion 10. When testing the file browser capability I was seeing the following error:

The server didn’t reply with a proper XML data. Please check your configuration.

The FCKeditor I was using relies on a ColdFusion connector that calls a custom function called FileUpload. This function is located in /editor/filemanager/connectors/cfm/cf_command.cfm. Since FileUpload() is a reserved function in ColdFusion 9+, the server was throwing a 500 error and the FCKeditor was displaying the less than useful error message above.

The fix was pretty easy. Just do a search and replace in /editor/filemanger/~. I replaced FileUpload with fckFileUpload. You will end up changing nine files total. That’s it. I probably should have simply updated the editor, but there were some custom js configurations that I didn’t want to track down. So… there you go.

Need a cheap SMTP host? Try Amazon SES

Written by Perry Woodin on . Posted in App Dev

I recently stood up a couple of ColdFusion servers on Amazon’s EC2. One of the servers is a staging server and I used localhost to send out mail. Turns out Amazon limits the amount of outgoing mail over port 25.

Of course, Amazon offers a solution in the form of Amazon Simple Email Service (SES). http://aws.amazon.com/ses/ From the Amazon SES page:

Amazon Simple Email Service (Amazon SES) is a highly scalable and cost-effective bulk and transactional email-sending service for businesses and developers.

I signed up for SES, and within 24 hours was accepted with an initial outgoing limit of 10,000 emails per 24/hrs. Not bad. And as you prove your worthiness (i.e. no spamming), Amazon ups the limit.

Jenkins ChromeDriver plugin Killing my Nodes

Written by Perry Woodin on . Posted in Deployment, Gotchas

I decided I wanted to learn how to use Selenium with Jenkins so I installed the ChromeDriver and Selenium plugins on Jenkins 1.486. Immediately after doing so, my Jenkins nodes started displaying the “Connection was broken” error message from the master Jenkins instance. Checking the connection from the node itself, everything looks fine. The node appears to be connected, but the master thinks otherwise.

I’ll obviously have to to some trouble-shooting. For now, I removed the ChromeDriver plugin and my node connections are working again.

Distributed Builds with Jenkins Nodes (master / slave setup)

Written by Perry Woodin on . Posted in Deployment

When I first started using Jenkins (it was Hudson at the time), I would push my ColdFusion updates to various servers (e.g. staging and production) with an FTP sync ant task. It wasn’t a particularly great way to do things, but it saved a lot of time and it worked. I used this setup because I wanted to manage all of the Jenkins projects from a single Jenkins point of entry. I wanted to avoid setting up and managing Jenkins on multiple servers. I have since abandoned the FTP sync in favor of ant running on each server via a Jenkins node.

This post is geared towards ColdFusion developers because it assumes you do not need to compile your code and simply want to update your code base with changes from SCM. I will also note that Jenkins nodes are often used to distribute work load. I’m not going to cover that here. I’m simply going to explain how I setup Jenkins nodes on a Windows 2008 server. I will write-up a more detailed post about the build process we use on some projects here at Troy Web.

If you don’t already have Jenkins running somewhere, you can go get a native installer at https://jenkins-ci.org/

Setting up a Jenkins Node on Windows 2008 Server

Log into Jenkins and go to Manage Jenkins => nodes.

  • Click New Node.
  • Give the node a name (e.g. Production), select Dumb Slave.
  • Click OK.

On the next page fill out the following:

  • # of executors. This controls the number of concurrent builds the node can run.
  • Remote FS root. This is the directory on your slave machine where Jenkins will install files necessary to run projects. Something like c:\Jenkins
  • Usage. Accept the default of Utilize this slave as much as possible.
  • Launch method. For Windows, select Launch slave agents via Java Web Start
  • Availability. Accept the default of Keep this slave on-line as much as possible.

With the new node defined, log into the master Jenkins FROM the slave machine and go to Manage Jenkins => nodes. Click on your node and you will be presented with a page showing details on how to launch the node.

I haven’t had any luck with the Launch button, so I recommend running the jnlp from the command line.

Once the node has connected to the master you should see a window that indicates the slave has connected.

To install as a window service, select File => Install as Windows Service.

Assigning Projects to a Node

If you want a Jenkins project to run on your new node, go to a project configuration page and check the box next to Restrict where this project can be run. In the Label Expression field, enter the name of the node where the project should run.

Troubleshooting the Connection Between Master and Slave

You may need to set anonymous read access for users in Jenkins.

I had to open up a port in the firewall and then set the port to fixed under Manage Jenkins => Configure System => TCP port for JNLP slave agents.

Distracto-boy meets layout=plain

Written by jbriccetti on . Posted in App Dev, Architecture, Misc Ramblings

I’m a huge fan of UI frameworks – particularly Twitter Bootstrap and jQuery UI – and I love making things look great – these frameworks really help me do that. Basically, I’m a wannabe designer. I admit it.

A simple bootstrap-infused login form

a bootstrap styled list view

Some days, i just spend way too much time on getting those buttons, just right, or the right table or grid layout – it can be a time-sucker and I sometimes find myself behind on a deadline because I spent too much time pushing around the UI and not enough time building the “application” code. Enter: layout=plain.

In the CF Wheels framework, a Rails-like approach to CFML development, the base controller class (/controllers/Controller.cfc) has an init method where most folks handle all the pre-processing needed for all requests. We scaffold our CF Wheels apps from our  TroyWeb1 base-build – which includes a “filter” in the base controller to handle  initialization and cleansing of request parameters as well as setting up defaults for the built-in params used by wheels for layout and format.

<cfscript>
	firewall();
	cleanseAndStrip(params);
	param name="params.format" default="html";
	param name="params.layout" default="";
	switch(params.layout) {
		case "plain":
			usesLayout("/plain");
			break;
		default:
			usesLayout("/bootstrap");
	} //end switch
	if(params.format EQ "xml" || params.format EQ "json"){
		get("cfsetting").setEnablecfoutputonly(true);
		get("cfsetting").setShowdebugoutput(false);
		set(showDebugInformation=false);
	}
</cfscript>

Like most app dev frameworks, CF Wheels leverages a front controller design pattern. That means all requests get routed through the same initial channel and that means certain things can easily be managed in that initial routing – like format and layout, which in CF Wheels are parameterized so the request can dictate what format needs to be returned. This is really great in the world of JSON feeds or other syndication streams. Thus, add ?format=json to the url and the plumbing to route your call to a json generating view is all built in. [checkout provides()] Layouts are managed this way too when combined with the usesLayout() controller method.

If you too are mesmerized by the shiny, candy-like buttons, join Distracto-boy in the fight. Create /views/plain.cfm with one line of code:

<cfoutput>#includeContent()#</cfoutput>

and simply change the default for params.layout to “plain”

	param name="params.layout" default="plain";

and there ya go. Badabing! all those distractions are gone and you can focus on creating the app without all that glitter.

layout=plain rendering of the same form

layout=plain on the same list view

No daddy

Written by jbriccetti on . Posted in Gotchas, Misc Ramblings

I was in the fray when a few clients called me today to say “help, the website is down” – it seemed odd a few sites were down, others weren’t. of course trying to get o the websites that were experiencing problems, well i just got nuthin’ – no error message back from the webserver or anything. seemed odd.

I saw the webserver instances were up so i added host headers for the IP quickly, just to see if i could get traffic routed there – and it worked – so i knew immediately it was a dns issue – and that’s when i discovered go daddy was down for the count.