Multipart form upload on Android

Android 1.5 includes Apache HttpClient 4 for the purposes of making HTTP requests. Unfortunately HttpClient 4 (or the version included with Android anyway) doesn’t appear to support multipart form uploads.

As it happens writing a solution from scratch (at the HTTP layer that is) is pretty straight forward. The main challenge is understanding how the POST request should be structured. Once you know that it’s simply a matter of putting all the pieces together.

The connection is managed using the java.net.HttpURLConnection class. There’s a few properties that need to set on the connection to ensure it can be written to and read from,

HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setDoInput(true);
conn.setUseCaches(false);

A special header informs the server that this will be a multipart form submission.

conn.setRequestProperty("Content-Type", "multipart/form-data; boundary=xxxxxxxxxx");

The “boundry” can be any string. In this example it’s xxxxxxxxxx. It’s used in the body of the request to seperate each field being submitted.

Next comes the main body of the request. Each field being submitted follows the same pattern. For example, to submit a text file called “helloworld.txt” with the contents “Hello World” and using the boundry string xxxxxxxxxx here’s what the request would look like,

--xxxxxxxxxx
Content-Disposition: form-data; name="filetoupload"; filename="helloworld.txt"
Content-Type: text/plain

Hello World
--xxxxxxxxxx--

The final -- is important. It tells the server it’s the end of the submission.

Here’s a full method that’s used to upload a file to a URL. It can optionally take a username and password which is used to perform BASIC authentication.

    public static void put(String targetURL, File file, String username, String password) throws Exception {

        String BOUNDRY = "==================================";
        HttpURLConnection conn = null;

        try {

            // These strings are sent in the request body. They provide information about the file being uploaded
            String contentDisposition = "Content-Disposition: form-data; name=\"userfile\"; filename=\"" + file.getName() + "\"";
            String contentType = "Content-Type: application/octet-stream";

            // This is the standard format for a multipart request
            StringBuffer requestBody = new StringBuffer();
            requestBody.append("--");
            requestBody.append(BOUNDRY);
            requestBody.append('\n');
            requestBody.append(contentDisposition);
            requestBody.append('\n');
            requestBody.append(contentType);
            requestBody.append('\n');
            requestBody.append('\n');
            requestBody.append(new String(Util.getBytesFromFile(file)));
            requestBody.append("--");
            requestBody.append(BOUNDRY);
            requestBody.append("--");

            // Make a connect to the server
            URL url = new URL(targetURL);
            conn = (HttpURLConnection) url.openConnection();

            // Put the authentication details in the request
            if (username != null) {
                String usernamePassword = username + ":" + password;
                String encodedUsernamePassword = Base64.encodeBytes(usernamePassword.getBytes());
                conn.setRequestProperty ("Authorization", "Basic " + encodedUsernamePassword);
            }

            conn.setDoOutput(true);
            conn.setDoInput(true);
            conn.setUseCaches(false);
            conn.setRequestMethod("POST");
            conn.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + BOUNDRY);

            // Send the body
            DataOutputStream dataOS = new DataOutputStream(conn.getOutputStream());
            dataOS.writeBytes(requestBody.toString());
            dataOS.flush();
            dataOS.close();

            // Ensure we got the HTTP 200 response code
            int responseCode = conn.getResponseCode();
            if (responseCode != 200) {
                throw new Exception(String.format("Received the response code %d from the URL %s", responseCode, url));
            }

            // Read the response
            InputStream is = conn.getInputStream();
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            byte[] bytes = new byte[1024];
            int bytesRead;
            while((bytesRead = is.read(bytes)) != -1) {
                baos.write(bytes, 0, bytesRead);
            }
            byte[] bytesReceived = baos.toByteArray();
            baos.close();

            is.close();
            String response = new String(bytesReceived);

            // TODO: Do something here to handle the 'response' string

        } finally {
            if (conn != null) {
                conn.disconnect();
            }
        }

    }

There’s nothing in this code particular to Android so it will work in any java application.

Bookmarks for 2010-02-14

The Pleasures of Counting by T. W. Körner
An interesting looking maths book. One to add to the “must read” list.

The Four Steps to the Epiphany by Steven Gary Blank
"Step-by-step strategy of how to successfully organize sales, marketing and business development for a new product or company."

Steak: How to Turn Cheap “Choice” Steak into Gucci “Prime” Steak
How to cook a really tasty steak using a *lot* of salt. I'll definitely be trying this one out.

Design 0.101 for Programmers
A nice step by step guide describing how to build a simple but nicely styled set of web pages. I usually use YUI or Blueprint to give me the basics but sometimes they can be a bit heavy if all you want is a simple page or two.

Fiddler Web Debugger
Useful proxy based tool for inspecting and "fiddling" with HTTP requests and responses.

Identifying Performance Issues on Android

While application performance is always in the back of your mind when developing PC applications it becomes much more important when developing on a mobile platform. The hardware is slower and the user expectations are much higher.

This problem came to the fore for me recently as I was porting UPM to the Android platform. Since UPM is a Java application I was able to reuse quite a bit of the code with little or no modification (all bar the SWING code actually).

During the porting process I found that opening the password database on the Android Emulator took about 12 seconds. This was a sub-second operation on the PC version so there was obviously a problem. Using Traceview, the profiling tool that comes with the Android SDK, the problem soon became apparent.

UPM stores all it’s data in an encrypted file. This file is read into memory where it’s decrypted. Here’s the code that was reading in the file…

FileInputStream fis = new FileInputStream(databaseFile);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int i = 0;
while ((i = fis.read()) != -1) {
    baos.write(i);
}
byte[] fullDatabase = baos.toByteArray();
baos.close();
fis.close();

Looking at it now it’s obviously bad code from a performance point of view as it’s reading in the file byte by byte. The reason I never replaced it was that it simply wasn’t a problem when running on a PC. The database file it reads is only a few kilobytes in size so it ran in under a second.

After identifying the bad code I replaced it with the following which runs in under a second.

InputStream is = new FileInputStream(file);

// Get the size of the file
long length = file.length();

// Create the byte array to hold the data
byte[] bytes = new byte[(int)length];

// Read in the bytes
int offset = 0;
int numRead = 0;
while (offset < bytes.length && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) {
    offset += numRead;
}

// Ensure all the bytes have been read in
if (offset < bytes.length) {
    throw new IOException("Could not completely read file " + file.getName());
}

is.close();

The lesson here is that Android, and mobile platforms in general, require you think a little bit more about elements of your design and code than you might normally.

Some useful Android links:
Designing for Performance
Designing for Responsiveness

Bookmarks for 2010-02-07

Dive Into Python 3
One of the better, if not the best resource for learning python.

South Dublin County Libraries
Homepage for the Tallaght and other south Dublin libraries.

Android Source Code Git Repositories
Looking for the Android source code? This is the Android Git homepage with references to all repositories. Useful if you’re developing an app and would like to see the source for some of the packaged apps.