Language Unification for In-House Application Development: Part II

Written by Sean Ryan on . Posted in App Dev

A Guide to selecting the most appropriate web language for your organization

In Part I of this series, we looked at the importance of investing some time into selecting a programing language for your organization and the types of questions that should be discussed with your group throughout the process. In Part II, we paint a fairly typical scenario of an organization in which the decision to rally behind one programming language has been made but the question of which language remains unanswered.

Part II

We start by listing the more popular Web programming languages today and work from there. The list is presented in alphabetical order and in no way represents a preference for one over another.

Note

The following table and brief explanations of “popular languages today” is a subset of the possible languages to use when writing web apps and to an extent, desktop apps. The list was carefully selected and excludes such languages as C/C++, Perl, Erlang, and Clojure/Lisp. This article assumes that if your organization is in consideration of language other than the nine languages that follow, its needs are outside of the scope of this article.

 

Language Use Programming Style Runs on Year/Age Frameworks
ASP.NET Web, server side scripting Object oriented CLR 2002/12 ASP.NET MVC, Spring.NET
ColdFusion Web, server side scripting Imperative/Procedural with support for Object oriented JVM, CLR 1995/19 CF Wheels, ColdBox, FuseBox, Mach-II, ModelGlue, some others
Groovy General purpose, client and server side scripting Object Oriented, functional, and imperative JVM 2007/7 Grails
Java General use; Desktop, Web, Client/Server Object oriented and imperative JVM 1995/19 Spring (specifically MVC), JSF, Struts, GWT, Tapestry, many more
Node.js (JavaScript) Client/Server, Web, Real-time applications, scripting Object oriented, imperative, and functional Node.js / Google V8 Engine 2009/5 Express, Geddy, some others
PHP Web, server side scripting Object oriented, functional, procedural, and imperative PHP Interpreter 1995/19 Laravel, Phalcon, Symphony2, CodeIgnitor, many more
Python General purpose, scripting, Web , Desktop Object oriented, functional, procedural, and imperative Various compilers, VMs, and interpreters 1991/23 DJango, Grok, some others
Ruby General purpose, Web Object oriented, functional, and imperative Various Ruby interpreters 1995/19 Rails
Scala General purpose Functional, object oriented JVM 2003/11 Lift, Slinky, Sweet, some others

 

Now that we have narrowed the field of options from seemingly hundreds to nine likely candidates, let’s review a realistic scenario for which an analytical selection can be made.

Scenario

You are the division Director of Information Technology for a medium sized government agency. This scenario applies equally as well for non-profits. The agency’s core mission is not building software but the division is an integral, and therefore critical part of the providing daily services required by the agency.  The software developed in-house is roughly 80% internal facing and 20% external facing. The internal community consists of 1,500 employees who use the custom software as part of their daily job duties whereas the external community comprises the entire world, yet more realistically about 20 million potential users. The developer pool is made up of about thirty programmers spread over six units/teams. The majority of programmers are familiar with Java, a small minority knows or understands the functional paradigm of programming, and everyone understands procedural/imperative as well as object oriented programming. Most internal applications are written in ColdFusion and all external applications have been developed in Java. Developer ages range from 18 to 55 but more than half are over 40.

You’re tasked with recommending to the Chief Technology Officer, a software development strategy that will 1) ease the recruitment of talented programmers by bringing the agency up to date with modern software development methods and by appealing to the up and coming programmer workforce, 2), reduce the agency’s technology fragmentation across each team and make assigning resources to inter-departmental projects easier 3) work within the existing network and server architecture with little or no modification, 4) take advantage of the existing agency talent to the largest extent possible, and 5) continue to meet or exceed the wide range of internal and external needs, from desktop applications to web portals, without impacting project deadlines and budgets.

Recommended Language and Framework

Groovy on Grails is our ultimate choice for the organization. Here’s how one might come to this conclusion.

Java

Java appears to be a natural fit. Given that there are already a number of applications running Java and ColdFusion, we know that some of the server infrastructure is already set up to deploy applications on a JVM. After all, ColdFusion is written in Java and runs on Tomcat.  We also know that most programmers in the organization know or are familiar with Java so it would be a lighter lift if we choose Java outright. Not having to train both programmer and system administrators also helps tip the scales toward Java. Following these arguments, it’s easy to lean toward one of the four languages that run natively on the JVM: ColdFusion, Java, Groovy, or Scala.

Python, PHP, and ColdFusion

We can objectively rule out Python and PHP based on the aforementioned arguments but both are perfectly acceptable language options when examined without the considerations of the scenario. Python in particular has garnered the attention and respect of web developers in recent years due to its robustness as a language.  Not only does it work well with frameworks for web applications, it can also be run as a scripting language on a variety of platforms, extending its usefulness as a tool for programmers. PHP and ColdFusion are effectively limited to the web world. Factoring in the need for web and non-web application development, languages that can be used on and off the web exhibit a more desirable quality.

.NET

ASP.NET can be ruled out based on the organization’s existing programming knowledgebase and on the existing support infrastructure. The .NET family of languages requires a Windows environment and while end users’ computers are mostly Windows, the servers are not. Introducing a new Windows infrastructure along with a new software infrastructure introduces additional technical and human resources costs. A switch to Windows-based development can be determined, for this scenario, time and cost prohibitive.

Node.js

Node.js isn’t as easily ruled out but when lined up with the remaining choices, one can start to chip away at it as the best option for the organization. For one, it’s very new and the available libraries are not as plentiful as the long established players like Java.  For another, Node apps are developed in JavaScript and JavaScript applications require a very particular style of programming. The applications process data asynchronously and rely heavily on closures, callback functions, and use a prototyping method of objection-oriented design. Those concepts can be a lot for some programmers to wrap their heads around and unless there is a compelling reason to overhaul the organization’s approach to designing software, it’s probably best to appeal to the widest skills market possible.

A Closer Look at Ruby, Scala, Java, and Groovy

Ruby, Scala, Java, and Groovy are all great contenders for the needs of this organization.

A Focus on Spring

If we select Java, the most robust framework is Spring. Spring is the most prolific Java framework series because it offers developers more than just a framework for web. There are eighteen projects that comprise Spring. They’re maintained by Pivotal Software, the same company that maintains Groovy and Grails. The framework has been so successful that it’s currently being ported to Scala as a project aptly named, Spring Scala. It operates as an IoC container, i.e., a dependency injection framework and follows the Front Controller pattern for which every resource request is first handed to Spring, pre-processed and subsequently handed off to a part of the application configured to handle that request. The exact same pattern is implemented in Groovy on Grails and in Ruby on Rails.

Due in no small part to the offerings of the Spring framework, we can rule out Ruby on Rails and Scala. Rails was an incredibly successful pioneer in the convention over configuration approach to development. Fortunately, the successes of Ruby on Rails lead to the creation of Groovy on Grails (and later, Scala) which follows the same principle but is Java-based. Most native Java code is also syntactically correct Groovy code, making the transition for a Java programmer very straight forward and much faster than learning a wildly new syntax. The Grails framework is in fact the Spring framework applied as a convention over configuration. Everything Spring offers is directly available in a Groovy on Grails application with no special integration necessary.

The out-of-the-box availability of both Java and Spring within a Groovy on Grails app, makes it an excellent fit given our scenario above. Spring was designed as a fast and flexible way to develop Java web apps. Groovy was inspired by Ruby and to address the short comings of Java. And finally, Grails is to Groovy what Rails is to Ruby. The end result is the best of all worlds while minimizing the disruptions caused by transitioning to a unified language platform across the organization.

Analysis

The scenario might read as an oddly specific set of circumstances. However, its goal is to represent the more general scenario of recognizing a need to have a consistent software development strategy; a strategy that harnesses an organization’s existing resources yet doesn’t alienate the future workforce it will need to recruit and come to rely on.

General-use vs. Nuanced Language Constructs

The fictitious organization above is far better served by selecting a general purpose language without getting into the nuances of each language’s constructs.

Each of the nine languages and their corresponding frameworks provide value to building out software. Some are faster for development (e.g., ColdFusion), some are more scalable (e.g., Scala), and still others offer very fast execution speeds (e.g., Node.js).

General-use languages like Java and Python, by their very nature, aren’t necessarily designed to solve a single problem exceptionally well. They appeal to a wider audience and allow for a variety of approaches to software design. General-use languages are excellent choices for organizations that need this sort of flexibility.

On the other hand Twitter for all intents and purposes has a single product – Twitter. Twitter’s font-end was once written in Ruby on Rails and their tweet storage written in Scala. In time Twitter moved away from Ruby on Rails in favor of Java and moved away from MySQL in favor of Lucene only to notice a 3x reduction in latency. Unlike the organization in the scenario, companies with a near singular focus – like Twitter – have the luxury of selecting a language based on very specific and pre-defined requirements.

Patterns and Frameworks

The organization in our scenario should focus on languages with several very popular frameworks that implement today’s proven patterns. Look for frameworks that provide structure to an application and encourage sensible design strategies without getting in the way of the developer. Convention over configuration is one way today’s frameworks offer a competitive edge and nearly all language-communities today have such an option.

It’s often helpful to understand the motivations for a language’s creation when deciding if it’s a language worth considering. Languages appeal to programmers for a variety of reasons. Possibly the largest factor is the speed and simplicity in which a program can be written.

Patterns of development help keep programmers focused on the business problem by laying out a proven method of attacking problems. Such patterns manifest in the form of frameworks. The Rails framework for Ruby was a pioneer in modern approaches to web development. The success of Ruby on Rails has been recognized and repeated in other frameworks. The Rails framework uses the Active Record pattern for example. This pattern greatly simplifies the pathway from the physical data model to the logical data model by coupling them directly through domain objects. This flies in the face of traditional patterns like the Transfer Object Pattern that stress a decoupling of models. Both patterns accomplish the same goal but with different layers of abstraction. Even the abstraction community giant, Java, has adopted this realization in various libraries like JOOQ and QueryDSL. Abstraction was once seen as beautiful, scalable, neatly organized code. However, abstraction and today’s need for rapid application development (RAD) have a seemingly perfect indirect relationship; not ideal. As the RAD approach to software development continues to grow in popularity, languages and frameworks need to adapt and keep up or be left behind in favor the new kids on the block. In short, change in software development is unavoidable; accept it and use it to your advantage.

Successful frameworks within and across languages copy the best parts of each other. Java inspired Scala, Ruby on Rails inspired the creation of Groovy on Grails. Rails also inspired the CF Wheels framework. When patterns work well, they’re incorporated into software design and when software is designed with standard patterns, there is a direct correlation with maintainability and in-house portability.

Language Ownership

Since the organization will have to build a large breadth of application types, it’s likely to require a wide range of libraries. It is highly recommended to select a language with a large and organized community driven code repository.

When the community is empowered to have a direct influence on a technology, be it a language or a framework, the technology has a much higher likelihood of long-term sustainability. Open source solutions have this inherently built right in but a project doesn’t necessarily have to be open source to be successful. Java for instance, while now open source, continues to rely on a Java Community Process for bettering itself. It brings together many perspectives and through diversity, creates a product developers want to use. The Apache Software Foundation is similar in this respect and its projects are all open source.

To once again use ColdFusion as an example, it’s not open source and community requests are sometimes implemented poorly or not fast enough to keep pace with competitors. This may be due in part to the road it’s taken through corporate acquisitions when Allaire was sold to Macromedia and Macromedia was sold to Adobe. At one time, ColdFusion was a singular focus of a company. Now, ColdFusion is just another product in a massive corporate portfolio (though, perhaps the same can be said of Java and Oracle).

Java has a massive library offering and it’s organized through a series of Maven repositories. Node.js, while still new on the scene, has the NPM library offerings. Most languages have something similar but not all. ColdFusion does not have any such organized library manager.

The Human Factor

It’s not always possible and practical to select a language for a project or for an entire organization based purely on market and textbook research. Doing so misses something very important: the human element. In part I of this article, I placed a notable emphasis on the programmers. Their existing skills, aspirations, and opinions matter just as much as any analytic findings. Programming is an equal mix of art and science. If a programmer is unhappy with his or her tools, the product and ultimately the organization, will suffer.

Language Unification for In-House Application Development: Part I

Written by Sean Ryan on . Posted in App Dev

A Guide to selecting the most appropriate web language for your organization

Part I

What language should our company use to develop in-house applications? A question that appears to carry with it a simple answer but too often, the decision isn’t well thought out or doesn’t originate from what makes sense for the organization as a whole.  It’s important to approach the decision from multiple points of view and to factor in more than just technology; team perspective is just as relevant.

This two part series was developed with Information Technology Management in mind with the hopes of stimulating a different way of thinking about language choice in an IT shop. Part I focuses of the process of selection. Part II breaks down a real-world scenario where an organization uses its team members’ current skill set to transition from a fragmented but mostly Java shop to a Groovy on Grails shop. While no one language is the right answer for all teams, Part II demonstrates an example of applying the principles discussed in Part I.

Introduction

The world of Information Technology is on a constant path of change and nearly all of it is out of the hands of any one organization. Organizations can opt to change with it, and continue to provide its customers with the data and information they demand or to not change and watch slowly as fewer and fewer customers turn to them for timely, relevant, accessible results.

Adopting to change is not just about maintaining a customer base. It’s also about being able to provide services that may be mandated by governing authorities, reducing the costs of processing requests, increasing the integrity input/output, providing expected or required levels of data protection, and controlling the cost to manage local IT services. A significant part of that cost is producing those services by building out in-house applications.

Once the decision has been made to develop custom in-house applications and a technology stack has been identified, the next step is to determine the most appropriate programming language for the organization or unit. Language selection should be driven by several factors and chief among them is the selection of frameworks.

Frameworks

Frameworks allow programmers to use a consistent pattern to receive, process, and produce data. They take the “busy work” out of app dev by handling events and processing parts of the request that need to be dealt with but are not directly part of the business problem.

For example, a framework should provide a way to secure a web application from unauthorized access into the system. It should allow a programmer to describe what “secure” means and what to do when a secured resource is requested. Spring security is an example of a web security framework that can be plugged into just about any Java application framework out there. Assigning a staff programmer to write the organization’s security framework is not only a waste of time; it’s also a risky idea.

A lot of smart people write and maintain frameworks and the community at large validates their usefulness by using them or not using them. Over the last twenty years or so frameworks have come and gone but several have stood the test of time and evolved into what has become a de facto requirement for building web applications.

Technological Costs

Generally, languages are free but their dependencies are not always free. For example, Adobe ColdFusion is cost $1,500 for a standard license and $8,500 for an enterprise license. However, it’s a full package solution. It ships with a pre-configured Tomcat application server and several admin tools, it runs on most major platforms, and is supported by Adobe. On the other hand Java is free. Tomcat, the most popular servlet container is free. GlassFish, the most popular application is free. Eclipse, the most popular development platform is free. Java’s runtime JRE (JVM) is free.  And finally, if running the app on some Linux distributions, the platform is also free.

Other costs include the tools necessary (or highly recommended) to write applications and to distribute application. Microsoft Visual Studio is needed to write C# and .NET applications. Distributing an iOS mobile app requires a yearly developer subscription to Apple, yet Android is completely free. One should be sure paying for a language solution is a necessary cost and in the best interest of the service that the organization will be providing. Sometimes the cost is necessary but it should be backed with quantifiable conclusions and not just antidotal observations from strong minded individuals.

Delivery of Applications

Desktop applications, web applications, network application, and mobile applications are very different and while they can all be written – to some degree – with most languages, language unification around all delivery mechanisms is a bad idea and any attempt to do so, would almost certainly manifest expensive problems shortly thereafter.

ColdFusion, for example is a relatively fast language to pick up but it’s a web application language designed exclusively for running small to medium sized web apps that use a traditional request/response paradigm. On the other hand, Java and .NET are both well suited for desktop, web, mobile, network, and just about any other use case. However, they are not as quickly learned and mastered and require external libraries/dependencies to accomplish all of the above.

Some languages are said to run on any platform. The quintessential example of this is Java, though a bit misleading. Java runs on any platform for which a JVM (Java Virtual Machine) exists. Since Java runs on a JVM and code must be further translated to the machine code it runs on, Java was viewed for many years as slow. This too is misleading since speed issues have long been resolved and are considered nearly inconsequential in today’s applications. However, for high performance network computing, C is a much better choice since it runs natively on the machine itself without any VM.  Another example is .NET. .NET applications require a Windows environment so if an organization’s infrastructure is mostly non-Windows, .NET is not a recommended choice.

It’s a good idea to choose a language that can be used for the majority of application types and one that can be run with the least amount of effort on most or all platforms.

Standards Based

The Web has been around for a little more than twenty years and web applications have been around almost as long. To say web applications have evolved in that amount of time is an understatement. Not only has the HTTP protocol changed significantly in that time, so too has the demand and expectations consumers have placed on web applications.

As more and more expectations were placed on the application, applications grew larger and more complex. Programmers trained (or not trained) in application development found themselves having to write scalable containers and complicated data pathways through their applications just to solve an otherwise simple business problem. Many were ill-equipped to write such applications and consequently some web apps lacked scalability and suffered from poor usability, buggy code, security holes.

As the software engineering community struggled to find the most effective techniques of working with this new challenge (web vs desktop) a set of standards and best practices emerged. Today, these well-tested solutions to common problems are implemented and shared with the community at large in the form of patterns, frameworks, and libraries.

Standards are critical to the success of an application, not only in producing the application, but also for the lifetime maintenance of the application. Today, developers are formally trained or otherwise learn the standard proven approaches to problem solving. Employing standards simplifies handing off a design from one developer to another and increases the predictability of the code. They also make debugging straightforward and reduce the likelihood of over engineering a solution and introducing a maintenance nightmare.

Industry and Developer Proliferation of the Language and its Frameworks

Perhaps the second biggest factor to consider when selecting a language is popularity. Honestly answering a few questions should weigh heavily on a language selection discussion.

How many [web|mobile|desktop|etc.] applications use this language?

The Java community likes to boast that over 3 billion devices run Java and they have good reason to. For better or for worse, it suggests that the language is pretty popular is likely to stick around for a while. Even if industry leaders – over night – were to tell the word Java is on its way out, the need for Java developers would not be impacted for a long time to come. After all, 3 billion devices and billions more applications would need to be maintained for at least some time. In reality, there is plenty of room for multiple languages at the forefront of industry.

How many developers in the region or your pool know this language? How long to bring a new developer up to speed?

A very important factor is availability of programmer resources. If migrating from one language to another, how many exiting team members know the new language? If hiring new developers, how prevalent are they in the community? If planning to train developers, how heavy of a lift is the new language to learn?

A programmer trained in functional programming may struggle with adjusting to an object oriented language or the MVC style of programming. Node.js is a JavaScript platform and JavaScript is a typeless language that relies heavily on closures.  Closures not easily grasped by some programmers. On the other hand Groovy is so close to Java that the learning curve for a Java programmer to learn Groovy is nearly flat and not very long.

How mature is the language? Is the adoption rate increasing or decreasing?

In the last dozen years, several popular web languages have popped up and have taken hold in certain segments of the community. Some examples include but are in no way limited to: Scala as a replacement for Java in the early 2000s that runs on the JVM, Rails (framework) in the mid 2000s as a convention over configuration web frameworks that runs on Ruby but with out the popular criticisms of Java, Grails (framework) shortly after Rails as a Groovy version with all the benefits of Rails but able to run on the JVM, Node.js in the late 2000s as JavaScript platform for building server side web apps in JavaScript.

As new languages and frameworks emerge, some languages struggle to keep up and begin to fade. For example, ColdFusion was a wildly popular for many years and to this day is still being actively used and maintained. However, its popularity is fading. It began as a procedural style language but to survive, it needed to adapt to add support for object oriented programming. By design, it never fully achieved that goal. When the RESTful style of web programming blew up in popularity as an important technique to decouple the front and back-ends, ColdFusion fell even further behind. When it finally added the RESTful feature, it was and continues to be plagued by bugs.

Programmers gravitate toward languages and frameworks that offer out-of-the-box solutions to the everyday problems they’re faced with so they don’t get in the way when solving the business problems they’re hired to solve. This cannot be understated when looking to hire talented programmers with the intent to keep them around.

Who owns the language? Is it being actively maintained? Are there add-on/plugins/libraries available? How many go-to frameworks exist for this language? How actively maintained is the framework?

This series of questions is to help answer the more general question: if the language can’t do it natively, is there a public distribution structure in place to use libraries that can do it? Many language-communities provide this feature. Java and Groovy have the Maven repositories, Node.js has the NPM repository, PERL has CPAN, Ruby has gems, Python has PyPi, and front-end JavaScript developers have Bower.

Without public support for sharing libraries and a mechanism for peer-reviewed / peer-tested code, programmers are left to reinvent the wheel time and time again. Valuable development time is wasted on developing common code. Programming is very much an art and like other artists, programmers enjoy sharing their creations with the community. Look for programming-communities where there is a groundswell of support for sharing their craft.

Committing to a Language and Framework

There are plenty of benefits to committing to a singe language or framework in an organization. Among the staff there is a consistent and predictable knowledge base. The ease of sharing or transitioning programmers from team to team is valuable for a project manager and management of a single infrastructure to support app distribution is valuable to a system administrator.

Language unification is not without its pitfalls either. It can lead to a lack of skill diversity among the organization’s programmers. It may also force a technology on a problem that may be better solved with a different language and without careful attention, it will impede of the intellectual growth of an organization’s programmers by reducing their professional exposure to other languages.

Weighing the pros and cons of selecting a common language should also take into account the purpose of the organization. A consulting firm is better served with a diverse body of languages and frameworks whereas a government agency is better served with a unified approach to app dev.

Using Grunt to Auto-Restart node.js with File Watchers

Written by Russell on . Posted in App Dev, Deployment, Front End, Productivity, Software

Preprocessors have become a very important part of the development life cycle. In the past we would just write some HTML, JavaScript and CSS with a backend and deploy a website.  Now for better speed, development experience and more manageable outcomes we have a multitude of languages that compile into these standards, e.g: CoffeeScript–> JavaScript, LESS –> CSS, Jade –> HTML… Then there is JS, CSS and HTML compression and minification after that.

Creating a development workflow to manage all of these transformations can be a very daunting task at the beginning of a new project. The complexity of the workflow can can also make or break a project. When ever I try to solve a problem I always try to look at the difficulty curve of reproduction and repeatability. In other words, it shouldn’t be difficult to setup a new developer on the project and it should be easy to perform 1k times a day without a work-performance loss.

Grunt has become a very important part of our  development life cycles. For compiled projects such as Java, adding in Grunt from ANT or Maven is relatively simple to in-line. For our node.js projects we wanted to be able to not only run the node server, but we wanted to be able to auto-restart the process while also auto-building resource files like LESS and JS files. Below you will find a lengthy Grunt file with commentary interlaced. This should help you get started with your own development environment.

'use strict';

module.exports = function(grunt) {
    grunt.initConfig({
        /**
            The concurrent task will let us spin up all of required tasks. It is very 
            important to list the 'watch' task last because it is blocking and nothing 
            after it will be run.
        **/
        concurrent: {
            dev: ["less:dev", "nodemon", "watch"],
            options: {
                logConcurrentOutput: true
            }
        },

        /**
            The nodemon task will start your node server. The watch parameter will tell 
            nodemon what files to look at that will trigger a restart. Full grunt-nodemon 
            documentation
        **/
        nodemon: {
            dev: {
                script: 'index.js',
                options: {
                    /** Environment variables required by the NODE application **/
                    env: {
                          "NODE_ENV": "development"
                        , "NODE_CONFIG": "dev"
                    },
                    watch: ["server"],
                    delay: 300,

                    callback: function (nodemon) {
                        nodemon.on('log', function (event) {
                            console.log(event.colour);
                        });

                        /** Open the application in a new browser window and is optional **/
                        nodemon.on('config:update', function () {
                            // Delay before server listens on port
                            setTimeout(function() {
                                require('open')('http://127.0.0.1:8000');
                            }, 1000);
                        });

                        /** Update .rebooted to fire Live-Reload **/
                        nodemon.on('restart', function () {
                            // Delay before server listens on port
                            setTimeout(function() {
                                require('fs').writeFileSync('.rebooted', 'rebooted');
                            }, 1000);
                        });
                    }
                }
            }
        },

        /**
            Watch the JS and LESS folders for changes. Triggering 
            fires off the listed tasks
        **/
        watch: {
            js: {
                files: ["client/resources/less/**/*.js"],
                tasks: ['copy:dev:custom'],
                options: { nospawn: true, livereload: true }
            },
            less: {
                files: ["client/resources/less/**/*.less"],
                tasks: ['less'],
                options: { nospawn: true, livereload: true }
            }
        },

        /** 
            Less task to compile LESS into CSS.
            Different options for dev and prod
        **/
        less: {
            dev: {
                options: {
                    compress: false,
                    yuicompress: false,
                    strictMath: true,
                    strictUnits: true,
                    strictImports: true
                },
                files: lessFiles
            }, 
            prod: {
                options: {
                    compress: true,
                    yuicompress: true,
                    strictMath: true,
                    strictUnits: true,
                    strictImports: true
                },
                files: lessFiles
            }
        },

        /**
            Used for production mode, minify and uglyfy the JavaScript Output
        **/
        uglify: {
            prod: {
                options: {
                    mangle: true,
                    compress: true,
                    sourceMap: true,
                    drop_console: true
                },
                files: {
                    'client/public/js/main.js': ['client/resources/js/main.js']
                }
            }
        },

        /**
            This copy tasks has two parts. The libraries will be rarely updated and are 
            only copied on startup. The custom sub-task will copy over application specific 
            JS since it doesn't need a preprocessor in dev
        **/
        copy: {
            dev: {
                custom: {
                    files: [
                        {
                            src: ["client/public/js/*.js"], 
                            dest: "client/public/js", 
                            expand: true, 
                            flatten: true
                        }
                    ]
                },
                libs: {
                    files: [
                      /** 
                        Array of file objects that reference bower libs }}
                      **/
                    ]
                }
            }
        }
    });

    /**
        Load all the GRUNT tasks
    **/
    grunt.loadNpmTasks("grunt-nodemon");
    grunt.loadNpmTasks("grunt-concurrent")
    grunt.loadNpmTasks("grunt-contrib-copy")
    grunt.loadNpmTasks("grunt-contrib-less")
    grunt.loadNpmTasks("grunt-contrib-watch");
    grunt.loadNpmTasks("grunt-contrib-uglify");

    /**
        Register tasks allowing you to run:
            grunt
            grunt run
            grun dev
            grun prod
    **/
    grunt.registerTask("run", ["concurrent:dev"]);
    grunt.registerTask("default", ["concurrent:dev"]);

    grunt.registerTask("dev", ["less:dev", "copy:dev"]);
    grunt.registerTask("prod", ["uglify:prod", "less:prod"]);
};

You will need to add a few requirements to your packages.json file to pull in the new requrements

    "devDependencies": {
          "open": "*"
        , "grunt-nodemon": "*"
        , "grunt-concurrent": "*"
        , "grunt-contrib-copy": "*"
        , "grunt-contrib-less": "*"
        , "grunt-contrib-watch": "*"
        , "grunt-contrib-uglify": "*"
    }

Using Bootstrap UI pagination links with jQuery Cycle Plugin

Written by jbriccetti on . Posted in App Dev

jQuery Cycle Plugin is a very popular “slider” widget that’s been around for many years. So long, in fact, that many of my old apps use it. Like, they’re pretty old now.

In those old days, I used jQuery UI for styling and while that had it’s challenges, it was easy to make the pager controls take on the themed “button” look:

 

jquery-cycle-jquery-ui-styling

<script type="text/javascript">
  $(document).ready(function() {
    $('.slideshow').cycle({
      fx: 'fade' // choose your transition type, ex: fade, scrollUp, shuffle, etc...
      ,pager:  '#pager'
      ,activePagerClass: "ui-state-highlight"
    });
    $("#pager a").addClass("ui-button ui-state-default");
  });
</script>

The ease of this, as you can see is that you simply have to setup all the link items that the cycle plugin builds in the pager div as “ui-button” and “ui-state-default” and then specify the active class for each one to be “ui-state-highlight”. An important thing to note is that structurally, the pager is just a bunch of anchor tags inside a div. That is as simple a structure as you can cook up. The pager div just starts out like this:

<div id="pager"></div>

For Bootstrap, the Pagination Widget is sweet, but requires a (better) more structured DOM – the link items have to be constructed using <li>’s; That’s not how jquery Cycle works, by default, but that’s easy enough to configure with the option (callback) pagerAnchorBuilder:

jquery-cycle-bootstrap-styling-1

<script type="text/javascript">
  $(document).ready(function() {
    $('.slideshow').cycle({
      fx: 'fade' // choose your transition type, ex: fade, scrollUp, shuffle, etc...
        ,pager:  '#pager'
        ,activePagerClass: "active"
        ,pagerAnchorBuilder: function(idx, slide) { 
          return '<li><a href="#">'+(idx+1)+'</a></li>'; 
        } 
      });
    });
</script>

So here, we’re “manually” constructing each pager item to take on the structure of an anchor tag inside of  a list item. With this, we setup the pager with a little more structure:

<div class="pagination pagination-small">
  <ul id="pager"></ul>
</div>

So far, so good. But here is the tricky part. We want to now use the bootstrap styles to make the “active” slide in the pager look, well, active (like we used “ui-state-highlight” in the jQuery UI example). That’s a little harder to do. as you can see from the above code & screenshot, just setting the activePagerClass option isn’t enough. The reason why this doesn’t work is because the cycle plugin will apply that style to the anchor tag, not the <li> tag. With Bootstrap’s Pagination widget, we need to apply it to the <li>.

{Enter Sandman Music} Checkout the option listed (1) up from the bottom on the jQuery Cycle options reference page. The updateActivePagerLink callback allows us to tweak how the class is applied, just like we were able to tweak how the pager link was built using the pageAnchorBuilder. The result?

jquery-cycle-bootstrap-styling-2

<script type="text/javascript">
  $(document).ready(function() {
    $('.slideshow').cycle({
      fx: 'fade' // choose your transition type, ex: fade, scrollUp, shuffle, etc...
        ,pager:  '#pager'
        ,pagerAnchorBuilder: function(idx, slide) { 
          return '<li><a href="#">'+(idx+1)+'</a></li>'; 
        }
         ,updateActivePagerLink: function(pager, idx){ 
          $(pager).find('li').removeClass('active').filter('li:eq('+idx+')').addClass('active'); 
        }
      });
    });
</script>

Eliminate jQuery Validation Errors in Eclipse

Written by Sean Ryan on . Posted in App Dev

Eclipse doesn’t like jQuery and each time I begin from a new install, I need to configure it to ignore validation errors on jQuery. I hate that little red x. See below.

Eclipse Validation Error for a jQuery library.

Eclipse Validation Error for a jQuery library.

 

Steps to Remove this Little Red X

  1. Select your project in the Package Explorer (or however you do it)
  2. Project…Properties
  3. In the left menu: JavaScript…Include Path
  4. In the source tab, expand the folder path that includes the jQuery problem. You’ll see that you probably are set to include all javascript and exclude none.
  5. Click Edit… on the right
  6. Click Add… in the exclusions
  7. Browse to the jQuery file. If you use a pattern, it may exclude other files so use your head and exclude only what you intend to.
  8. Click Apply.

 

No more little red x

No more little red x

SSH Keys and ssh-agent on Windows/Linux/Mac

Written by Sean Ryan on . Posted in App Dev

Why am I Writing This

Using SSH to securely connect to remote servers is well understood by anyone needing to do it. However, when you ask people to use a public/private keys to connect, it starts to get a little murkier – mainly because the process is not well understood by some folks. This article breaks down the confusion and simplifies the process to a point where you’ll wonder why you haven’t been doing this all along.

Standard SSH

Let’s assume you need to connect to server1.domain.com as user superman. You can do this with the following command.

ssh superman@server1.domain.com

This approach is fine but it involves sending your password to the server for authentication and there is no verification of who you are. Consequently, anyone with your password can get in and eavesdropping on your connection is a possibility, albeit difficult.

SSH Keys

SSH keys use the public/private key paradigm to verify the authenticity of a connection and your password is never sent to the server you’re connecting to. The private key is secret, known only to you, and stored on the client whereas the public key is well known and provided to the server. When a connection attempt is made, a challenge/response routine takes place that authenticates who you are and sets up a secured connection. This is more secure than a password and greatly reduces/eliminates the possibility of a successful brute force attack.

In fact, SSH servers can be configured to only allow communication through the use of public/private keys; AWS Linux EC2 servers default to this behavior. I’ll show how this can be done at the end of the article.

Environment

For the sake of this article, I assume you are working from a command-line interface. On Windows, I assume you are working from within cygwin. Although, the concepts are easily applied to however you are connecting.

The first thing we need to do is make sure you have the ability to create SSH keys.

On Windows

Best option: Download/install cygwin and choose OpenSSH from the package list.

Other option: download/install OpenSSH as its own application

On Mac OSX / Linux

Turn your computer on.

From a command prompt, type ssh and then hit tab twice. You should see the ssh programs we’ll be working with:

  • ssh
  • ssh-add
  • ssh-agent
  • ssh-keygen

Mac OSX

Windows

Create Your Key Pair

Creating a key pair is very simple but there is one very important guideline to remember – though not required, you should include a passphrase on your private key. Failing to use a passphrase lowers your effective security. Your private key is used to unlock encrypted messages. It’s akin to leaving the key to your house under the mat. Adding a passphrase is akin to locking your key in a lock box and putting it under the mat. I say under the mat because the typical location of these keys is pretty standard.

Weak Argument Against Using a Passphrase: I’ll have to enter a password every time I use SSH and I don’t want to :(

My rebuttal: Use ssh-agent and stop complaining.

To create your public/private key pair on a command line, run the following command.

ssh-keygen

The default algorithm used to create the keys is RSA. We’ll stick with defaults here. When prompted, enter a passphrase (or don’t – your choice). You should see output similar to the screen shot below.

Mac OSX ssh-keygen

You can also use a putty program on Windows called puttygen.exe. To use puttygen to generate your keys, open the program and click Generate key pair… from the Key menu.

I love the random number generation here!

You can add a passphrase and save both keys after the randomness is done.

That’s it. Now we need to share the public key with the server we’ll be connecting to. You can get the file to the server anyway you want.

HowTo: Install a Public Key on a Linux Server

Here’s a one-liner that’ll do it.

cat ~/.ssh/server1.pem.pub | ssh superman@server1.domain.com 'cat - >> ~/.ssh/authorized_keys2'

This pipes the content of your public key to SSH which in turn connects to your server. Once the connection is established, it takes the contents of standard in (result of piping content through ssh) and appends it to a file called authorized_keys2 in your account’s .ssh folder.

Note: You will be prompted for a password. This is your standard ssh password and should be the last time you’ll need to enter it in this context. Also, if .ssh directory does not exist, this command will not create one. You’ll need to do that yourself and then re-run the command.

HowTo: Install a Public Key on a Windows Server

This is a bit trickier, but only because we need to know what ssh server the Windows instance is running.

I’m Assuming WinSSHD

If you’re connecting to WinSSHD on Windows, you’ll need to import your account’s public key into the application using the graphical interface. From the control panel of WinSSHD, click “Open easy settings” from the settings area.

Control Panel of WinSSHD

For WinSSHD Virtual accounts, click the third tab – the one named Virtual Accounts. If you connect to WinSSHD using Windows accounts, click the second tab – the one named Windows Accounts.

In either case, find your username, and click the edit icon in the keys column, then click Manage. You can import your key from this screen. After you import, you’ll see your entry in the list.

Keys list in WinSSHD

Connect to the Server Using Your SSH Key

The next step is using your SSH client to connect. From a command line, run the following command.

ssh -i ~/.ssh/server1.pem superman@server1.domain.com

In putty, open SSH from the Connection drop down and then click Auth. You’ll be able to import your private key there.

Important for Putty users: if you didn’t use puttygen to generate your private key but you want to use putty for your SSH needs, you’ll need to convert your private key into a .ppk file. This is simple. Open puttygen and select Import key from the Conversions menu. Browse to your private key and and enter its password (if you supplied one). You can then save the private key in the .ppk format that putty expects. You don’t have to bother with the corresponding public key.

When you make the connection you will be prompted for a password but it is not the account password.  The password you are prompted for is for unlocking your private key. Enter it correctly and SSH will use the key to communicate with the server having never sent your actual password to the server.

Stop Repeatedly Entering Passwords

We could stop here and feel good about augmenting our connection security but we can still simplify our lives further by reducing the number of times we’ll need to enter the key’s password.

 Mac OSX

On a Mac, this really couldn’t be simpler. Not only is ssh-agent started each time your computer starts up, it also loads into itself any SSH keys it finds in your keychain. Further, when connecting over SSH to a server using a key that isn’t in the keychain, keychain will pop up and ask for the password and [optionally] store it for you. If you choose to store it, ssh-agent will never prompt you for it again. Instead, it will just get it from keychain whenever it needs it. Check it out.

By running the following command, I’m prompted to store the password in keychain.

ssh -i ~/.ssh/superman.pem superman@server1.domain.com

 

Since ssh-agent is already running, it will get the password from the keychain when I attempt to connect. This is a Mac-Specific feature and really sweet.

Now that the server’s identity has been added to ssh-agent (automattically), I never have to point to my private key again.

ssh superman@server1.domain.com

…will work without [prompting for] a password now.

Linux / Cygwin

ssh-agent still works fine with Linux and cygwin but it’s a bit more manual.

First, you need to start the ssh-agent by running the command ssh-agent.

Run the following command to add your private key identity to the ssh-agent.

ssh-add ~/.ssh/superman.pem

The screen shot below describes what’s happening.

ssh-add -D removes all identities from the ssh-agent.

ssh-add -l lists all identities in the ssh-agent.

ssh-add superman.pem add the private to the ssh-agent (notice I’m inside my .ssh directory)

ssh-add -l shows the newly added identity.

Once added, I can now ssh to server1 using the standard ssh connection string:

ssh superman@server1.domain.com

Loading ssh-agent at Startup

It’s important to note that ssh-agent is session-based. It only stores identities during the current session. As soon as your computer is shutdown/rebooted, the agent no longer contains identities.

Mac OSX: ssh-agent is started and loaded automatically. You don’t need to do anything else.

 Linux

Some distros will start ssh-agent on start up. To check if you’re currently running ssh-agent, run this command:

ps aux | grep ssh-agent

If it’s running, you’re good and don’t need to do anything else. If it’s not running, you can put the following bit of shell code in your .bashrc file.

 

Cygwin

 

Coming soon…

Using Inner-Classed Enums with ColdFusion and Java

Written by Russell on . Posted in App Dev, Gotchas

I recently had to call a method from Java that required an Enum type. In my case I did not have control over the java code and the Enums were stored inside a Class called Enums. It was not readily apparent to me on how to access these inner-classed Enums and there wasn’t any specific documentation that I could find on the topic. I struggled a bit with trying to make a string work (the true end result), using JavaCast and trying to instantiate the class and Enum directly. Thankfully the final solution was actually easier than expected.

First we need to get the Enum from the Java Class.
<cfset local.FULL = createObject("Java", "com.my.path.datatypes.Enums$RegistrationResultsFormat").FULL />

Pay close attention to “$RegistrationResultsFormat”. This resolves to my Enum “public enum RegistrationResultsFormat { … }”.

If you perform a CFDUMP on the results of the createObject you will get a list of your possible options. In my case, RegistrationResultsFormat had FULL, COURSE and ACTIVITY. It is important to note that a CFDUMP of both the core createObject as well as the results of .FULL are exactly the same. Do not fret, you are still getting the desired end result and you may use it as is.

Next we need to pass the Enum value into that Java method. In my example it is the second argument.
<cfset local.results = anotherJavaMethod( javaCast("string", arguments.id), local.FULL) />

In my actual use case, I stored this result in the variables scope of my component and got the value inside the init method.

Front-End Resource Roundup

Written by tphillippi on . Posted in Front End

Note: this is a list of resources I’ve used for clients in the past to help other developers beginning to learn front-end tools get up to speed


Here is a list of resources for you to use to become more familiar with html, css & javascript. The resources are categorized  by the amount of time or difficulty required for adequate digestion of the information. The easiest/shortest is surfing, resources you can skim through in minutes as you need them. Next is snorkeling, resources better digested over a period of about an hour or more to really go through Lastly, deep diving, resources that cover topics in depth in several hours or more (this doesn’t mean you can’t pick and choose chapters or sections as you need them, though!). 

General Resources:

HTML (Structure):

CSS (Appearance):

JavaScript (Functionality): 

UX/Usability/Best Practices:

Some Tools:

Don’t be intimated by the vastness of the resources outlined here. We suggest for each topic you want to dive in to, take a look, see what appeals to you, and jump in. Then, over time, periodically refer to the deep diving sections to further your skills. I hope this is helpful! Enjoy!

LESS: Resource Roundup

Written by tphillippi on . Posted in App Dev

The resources in this post accompany a brief presentation, slides here: http://techblog.troyweb.com/wp-content/uploads/2013/04/Less.pdf

This will be growing over time as I add to it.

How to Compile LESS

In addition to using CodePen.io, there are various client side tools to compile your LESS for use in projects.

Mac

  • CodeKit ($25) – Please, just do it. If you’re on the Mac, this app is for so much more than LESS.
  • Live Reload ($10) My understanding it is quite similar to Codekit but not as full featured
  • LESS.app (FREE!) By the same guy who does Codekit. LESS.app evolved into Codekit.
  • Simpless (FREE!) Simple tool for compiling, also allows minification.

Windows

  • Simpless (FREE!) Simple tool for compiling, also allows minification. I used this when I was a Windows user.
  • WinLESS (FREE!) I’ve also used this. Doesn’t offer simplicity of other tools, but has more robust error reporting.

Where to Learn More

Adding a Polygon Search to CFMongoDB

Written by Sean Ryan on . Posted in App Dev

CFMongoDB rocks my socks…when I need to use ColdFusion.

I noticed that the CFMongoDB doesn’t natively support a polygon search :( It’s ok though because I wrote one and as far as our team’s testing is concerned, it works.

I added a method to DBCollection.cfc directly under the find method called findWithin_Poly. It follows.

 

function findWithin_Poly(string loc, array poly){
  polyConverted = CreateObject("java","java.util.ArrayList").init(ArrayLen(poly));
  for ( i =1 ; i lte ArrayLen(poly) ; i++){
    polyConverted.add(poly[i]);
  }
  polyObject = variables.mongoConfig.getMongoFactory()
              .getObject("com.mongodb.BasicDBObject").init("$polygon", polyConverted);
  within = variables.mongoConfig.getMongoFactory()
              .getObject("com.mongodb.BasicDBObject").init("$within", polyObject);
  query = variables.mongoConfig.getMongoFactory()
              .getObject("com.mongodb.BasicDBObject").init(loc, within);
  var search_results = [];
  search_results = collection.find(query);
  return createObject("component", "SearchResult").init( search_results, structNew(), mongoUtil );
}

As you can see, the process is very straight forward. I build up a BasicDBObject according to the Mongo Java API and call a slightly different find method on the Mongo collection.

Parameters

The loc parameter is a string that represents the key of the document that holds the coordinates of the point. I used loc because that what I saw all over the documentation but it’s clear about not needing to be called loc.

The poly parameter is an array of arrays. Each element of the poly array is itself an array of exactly two elements: an x and a y. It represents a list of points which describe the boundaries of the polygon.

Format

The format of the poly parameter is important – as is the data in the collection. CF will automatically make your array elements strings, but they need to be doubles – java doubles.

To do this, cast them with javacast(). See the example below.

 

poly = ArrayNew(1);
polyA = ArrayNew(1);
polyA[1] = javacast("double",3);
polyA[2] = javacast("double",3);
polyB = ArrayNew(1);
polyB[1] = javacast("double",8);
polyB[2] = javacast("double",3);
polyC = ArrayNew(1);
polyC[1] = javacast("double",6);
polyC[2] = javacast("double",7);
ArrayAppend(poly,polyA);
ArrayAppend(poly,polyB);
ArrayAppend(poly,polyC);

You would then use this to call the  findWithin_Poly function.

hits = people.findWithin_Poly("loc",poly);

Now, I haven’t done this yet, but I would recommend doing it: add a utility method that would cast all of the elements to double. Simple and reduces clutter.