Javascript application development in Mobile devices

Build Native Applications using Javascript

Palm’s new OS is the first mobile platform to be built from the ground up to combine standard technology, innovation and integration. At its core, webOS leverages several industry-standard technologies, including web technologies such as CSS, XHTML and JavaScript. You can think of webOS applications as native applications, but built from the same standard HTML, CSS and JavaScript that you’d use to develop web applications. Palm has extended the standard web development environment through a JavaScript framework that gives standardized UI widgets, and access to selected device hardware and services.

Javascript can be used on the iphone too.  The HTML 5 specification provides a new mechanism for client-side data storage: JavaScript database support. This feature shipped originally with iPhone OS 2.1. When you use “Add to Home Screen” from the “+” button on Safari for iPhone, a web application with a manifest defined (per the HTML5 spec) will be saved with any cached resources.  It is also worth noting that these features have not been added to Safari on Mac OS X or Windows.

Desktop Application development using Javascript

Adobe AIR is a cross-platform desktop runtime created by Adobe that allows web developers to use web technologies to build and deploy Rich Internet Applications (RIAs) and web applications to the desktop. This means what you can write you application using Javascript and then let AIR to figure out how to run your code in Windows, Linux, Mac and future operating systems.

Desktop App using Javascript

The application shown in the pic is written using Javascript, HTML and CSS. It uses the ExtJS library. Javascript can be used to dock the application in the taskbar, delete and modify files on the hard drive, communicate to different server, use AJAX, etc. You can even use flash within your app to make the application look and feel much better. This gives the web developer good access to a desktop.

Server side Scripts using Javascript

Server side JavaScript has been around for a long time and potentially offers some unique and interesting advantages over other languages (like PHP, ASP, etc) because the same language is spoken by both client and server. There are numerous attempts to bring server side javascript to the masses.  Server side Javascript has to do much more than what it has to do at the browser level with access to databases, files, and networking, as well as logging, process management, scalability, security, integration APIs, and extensibility. You can even use your own javascript libraries on the server jQuery, dojo, Ext JS, prototype, etc.

Javascript in other places

- Adobe reader (from v3.02) supports Javascript for forms and basic operations. SOAP support has also been included since version 7.0. There is also access to the entire 3D Javascript API. More details on using Javascript is available here
- Open office application suite supports Javascript to write macros. This move looks like a sensible addon given the usage of VBScript in MS Office suite.
- Apple’s Dashboard Widgets, Microsoft’s Gadgets, Yahoo! Widgets, Google Desktop Gadgets are implemented using JavaScript.
- The oddest one in this list will have to be the  Philips Remote Control which uses Javascript

ECMAScript 4 (ECMAScript Harmony)

Javascript has been relatively stable since the third edition of ECMAScript published in 1999. Features under discussion for a future edition originally ECMAScript 4 now ECMAScript Harmony include Classes, a module system, static typing and Optional type annotations, Generators, Iterators, Destructuring assignment, algebraic data types. Well if you think this is ambitious you should see the original ECMAScript 4 which had more semantic and syntactic innovation. Packages, namespaces and early binding from ECMAScript 4 are no longer included for planned releases with ECMAScript Harmony. The intent of these features is partly to better support “programming in the large”, and to let programmers sacrifice some of the script’s ability to be dynamic for performance.

So yes as a web developer who uses Javascript on a daily basis, I am excited to watch Javascript grow and be available in more places.

Standard Javascript file include method

<SCRIPT src="A.JS" language="JavaScript/text"></SCRIPT>
<SCRIPT src="B.JS" language="JavaScript/text"></SCRIPT> 

<IMG src="1.GIF" />
<IMG src="2.GIF" />
<IMG src="3.GIF" />

Javascript blocks the other elements from loading (Example)

Script DOM element method

var p = g.getElementsByTagName("HEAD")[0];
var c = g.createElement("script");
c.type= "text/javascript";
c.onreadystatechange = n;
c.onerror = c.onload = k;
c.src = e;
p.appendChild(C);

Javascript executed without blocking any element from loading

This method creates a DOM element for each Script and then adds the element to the HTML. 

NOTE: 

1) Only when the appendChild function is called the Javascript will be executed.
2) For inlined code that depends on the Javascript and also for a similar method of Asynchronous Script Loading – See Steve Souders blog
3) If you go to MSN.com (Alexa top 10 site based on traffic) and hit view source you can see the javacript elements are included by using the script dom element to load the web pages faster. This is a good example of where you can use the script dom element method for certain js files that do not have inlined code dependency. Let us run a Pagetest waterfall report and you can see the following for MSN.COM

Pagetest Waterfall report of MSN.COM shows no blocking during page load

The problem

Once hackers get into your website either by exploiting known vulnerabilities in any of the installed programs OR by getting FTP access to your server, the first thing they usually do is to plant backdoor scripts to log them in again at a later date. They need some executable script on the server to gain access to MySQL passwords, installation passwords or even edit settings in your wordpress or other installations.  We have also seen situations where the site was left largely unchanged except for malicious javascript code added to the bottom of the index.php or index.html files.

So in short the bad guys have taken over your server and running anything from a backdoor script or launching phishing attacks or sending tons of spam emails. You will not know that your server is hacked until you get blacklisted on spamhaus or your customers get redirected to some random site or worser still when you are contacted by ebay/paypal/some bank saying that your website is phisihing their customers. The problem is that we dont even have an idea that our site is hacked until it is too late or too embrassing.

Simple Solution – Website Change Detection System

We need a script on the server that detects any changes or to any executable file on the server or any new file on the server from HTML, JS, to PHP, ASP, Perl, Python files etc.  If we generate a hash value of all our files and then compare them periodically, then we will be able to detect when our codebase has changed on the server.

These are the steps that our change detection system performs (It takes about 500ms to execute on a typical server):

1. Load configuration file (contains password, exclude list)
2. Check password from request before starting (recommended)
3. Recursively run through every file and sub folder on the server within the current directory of the script.
4. Generate a hash for each file and arrive at the master hash.
5. Compare master hash with hash the user has and alert if different!

Notes:

• This script at this simple level is almost 100% fool proof in detecting changes to the codebaes give that the hacker or bots don’t know of websiteCDS presence.
• At this stage the script cannot detect SQL injection attacks and changes to code that are saved to the database.
• The users hash is not stored on the server at any time, the comparison with master hash can be done at cron script level or using siteup as discussed below.

We have the following code written in PHP but you can do the same with any other scripting language to perform similar checks. We have started the project under google code and is available here: PHP code for WebsiteCDS
See the readme file in the download for help with setup.

Different ways to automate the alert system

Method 1: Using our trusted cron job
A cron job can be setup to run the website CDS, compare the results with the last known valid hash and send out an email alert.

Method 2: With Siteup
Siteup is a free tool for windows systems that can be set to periodically check if your website is reachable. This is recommended for those of us who don’t want to setup a cron job. It can be downloaded here http://www.xequte.com/other/index.html#siteup We can use this to frequently call our change detection system and then use the siteup word search to check if the hash value is the same as what we have from our last codebase update. ( See screenshot )

NOTE: The project & code included is the first version of the change detection system and kindly submit your ideas and comments here or an issue or feature request in google code project for websitecds.

Example of a correctly formatted email address but still wrong:

mailbox.does.not.exist@reddit.com [VALID email format but it does not exist]

Above case specifically happens when you take important customer email on phone and you type in the wrong email. So is there a QUICK solution to really check the email without sending a test message to the user? Yes.

The solution

A quick & simple check below can be implemented in most programming language including PHP, Python etc. It relies on using the same SMTP which is used to send emails.

To check if user entered email mailbox.does.not.exist@reddit.com really exists go through the following in command prompt.

First - Find mail exchanger of reddit.com

COMMAND:
nslookup – q=mx reddit.com
RESPONSE:
reddit.com      MX preference = 10, mail exchanger = mail.reddit.com
mail.reddit.com internet address = 208.96.53.70

Second - Connect to mail server mail.reddit.com

COMMAND:
telnet mail.reddit.com 25
RESPONSE:
220 mail.reddit.com ESMTP Postfix NO UCE NO UEMA  C=US L=CA Unsolicated electronic mail advertisements strictly prohibited, subject to fine under CA law CBPC 17538.45.  This electronic mail service provider’s equipment is located in the State of California.  See http://www.reddit.com/static/inbound-email-policy.html for more information.

COMMAND:
helo hi
RESPONSE:
250 mail.reddit.com

COMMAND:
mail from: <youremail@gmail.com>
RESPONSE:
250 2.1.0 Ok

COMMAND:
rcpt to: <mailbox.does.not.exist@reddit.com>
RESPONSE:
550 5.1.1 <mailbox.does.not.exist@reddit.com>: Recipient address rejected: User unknown in local recipient table

COMMAND:
quit
RESPONSE:
221 2.0.0 Bye

NOTES:

1) the 550 response indicates that the email address is not valid and you have caught a valid but wrong email address. This code can be on the server and called on AJAX when user tabs out of the email field.  The entire check will take less than 2 seconds to run and you can make sure that the email is correct.
2) If email was present the server will respond with a 250 instead of 550
3) There are certain servers with a CATCH ALL email and this means all email address are accepted as valid on their servers (RARE but some servers do have this setting).
4) Please do not use this method to continuously to check for availability of gmail / yahoo / msn accounts etc as this may cause your IP to be added to a blacklist.
5) This is to supplement the standard email address javascript validation.

Telnet screenshot in windows – Check email using SMTP commands

UPDATE: PHP code added on 26th January 08

1) SMTP check code in PHP – DOWNLOAD

2) Usage example - DOWNLOAD

CAPTCHA is an acronym for “completely automated public Turing test to tell computers and humans apart.” This can be with images / audio or whatever we will see in future.

Why do sites use it?

CAPTCHA is used to prevent bots from automatically submitting forms with SPAM or other unwanted content. Google and other companies use it to prevent bots from creating multiple Gmail accounts.

What is wrong with it?

Users will have to enter this additional information every time they have to submit a form. This is getting more and more difficult to decode for humans as the bots are getting better at it. I am sure that everyone reading this post has atleast once got a CAPTCHA entered wrongly.

Alternative Simple soultion:

NOTE: Does not apply for highly targeted sites like gmail, yahoo mail and others alike.

However,
If you have a sales form which requires an entry of username, email and phone number for a call back
OR
a simple contact us form with just name, number, description.

1) Have server validation of data.
All the forms have only client validation in javascript. Just validate in server and you can have 70% of spam bot submissions caught.
EG: If your form has Name, Email, Telephone.
The bot will send some 500 character text in Name and valid email and some random data in phone field. A simple validation on the server to trash Name having more than 30 characters will do the trick.

2) Hidden input element
Add an extra text input element to your form. In an external style sheet you set the element to display: none; thus making it invisible to all users with CSS enabled. Spam bots will usually fill all fields in a form you know that any forms submitted where this invisible field is not empty are spam.

With the above two simple steps you can see that most sites can avoid spam messages and still not having to use a captcha.

So in short – for all the websites with simple contact forms why do we use CAPTCHA and risk giving the customer an additional field to fill and risk not to getting them to fill it at all ??