Sunday, February 6, 2011

A servlet filter for HTTP PUT (and network cameras)

I purchased an IP camera recently to use and test for near-real time weather pictures on the WeatherCurrents.com web site. After a fair amount of research, the model I chose was a Panasonic BB-HCM511A, which provides low-light color images at night, and power-over-ethernet (POE) capability for simplified cabling.

One of the web camera's features is the ability to push out images at regular intervals over the internet using FTP or HTTP. Since I don't want to maintain a FTP server for WeatherCurrents, I chose HTTP. The screen allows configuration of a URL, and a username and password (for basic HTTP authentication).

It doesn't allow for TLS/SSL, unfortunately, which I consider to be a security mistake.

When I tested posting an image to Apache Tomcat (version 6.0.32), I found it was attempting to upload the image with an HTTP PUT. The PUT, DELETE and TRACE commands are built in to the default servlet but are off in the default configuration.

To turn them on in Tomcat, the web.xml file in the conf directory would need to be modified:


  <!-- The default servlet for all web applications, that serves static     -->
  <!-- resources.  It processes all requests that are not mapped to other   -->
  <!-- servlets with servlet mappings (defined either here or in your own   -->
  <!-- web.xml file.  This servlet supports the following initialization    -->
  <!-- parameters (default values are in square brackets):                  -->
  <!--                                                                      -->
  <!--   debug               Debugging detail level for messages logged     -->
  <!--                       by this servlet.  [0]                          -->
  <!--                                                                      -->
  <!--   fileEncoding        Encoding to be used to read static resources   -->
  <!--                       [platform default]                             -->
  <!--                                                                      -->
  <!--   input               Input buffer size (in bytes) when reading      -->
  <!--                       resources to be served.  [2048]                -->
  <!--                                                                      -->
  <!--   listings            Should directory listings be produced if there -->
  <!--                       is no welcome file in this directory?  [false] -->
  <!--                       WARNING: Listings for directories with many    -->
  <!--                       entries can be slow and may consume            -->
  <!--                       significant proportions of server resources.   -->
  <!--                                                                      -->
  <!--   output              Output buffer size (in bytes) when writing     -->
  <!--                       resources to be served.  [2048]                -->
  <!--                                                                      -->
  <!--   readonly            Is this context "read only", so HTTP           -->
  <!--                       commands like PUT and DELETE are               -->
  <!--                       rejected?  [true]                              -->
  <!--                                                                      -->
  <!--   readmeFile          File name to display with the directory        -->
  <!--                       contents. [null]                               -->
  <!--                                                                      -->
  <!--   sendfileSize        If the connector used supports sendfile, this  -->
  <!--                       represents the minimal file size in KB for     -->
  <!--                       which sendfile will be used. Use a negative    -->
  <!--                       value to always disable sendfile.  [48]        -->
  <!--                                                                      -->
  <!--   useAcceptRanges     Should the Accept-Ranges header be included    -->
  <!--                       in responses where appropriate? [true]         -->
  <!--                                                                      -->
  <!--  For directory listing customization. Checks localXsltFile, then     -->
  <!--  globalXsltFile, then defaults to original behavior.                 -->
  <!--                                                                      -->
  <!--   localXsltFile       Make directory listings an XML doc and         -->
  <!--                       pass the result to this style sheet residing   -->
  <!--                       in that directory. This overrides              -->
  <!--                       contextXsltFile and globalXsltFile[null]       -->
  <!--                                                                      -->
  <!--   contextXsltFile     Make directory listings an XML doc and         -->
  <!--                       pass the result to this style sheet which is   -->
  <!--                       relative to the context root. This overrides   -->
  <!--                       globalXsltFile[null]                           -->
  <!--                                                                      -->
  <!--   globalXsltFile      Site wide configuration version of             -->
  <!--                       localXsltFile This argument is expected        -->
  <!--                       to be a physical file. [null]                  -->
  <!--                                                                      -->
  <!--                                                                      -->
    <servlet>
        <servlet-name>default</servlet-name>
        <servlet-class>org.apache.catalina.servlets.DefaultServlet</servlet-clas
s>
        <init-param>
            <param-name>debug</param-name>
            <param-value>0</param-value>
        </init-param>
        <init-param>
            <param-name>listings</param-name>
            <param-value>false</param-value>
        </init-param>
        <init-param>
            <param-name>readonly</param-name>
            <param-value>false</param-name>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>



If enabled, this would have the effect of opening the floodgates to PUTs and DELETEs, and only careful control over the PUT and DELETE methods through <security-contraint> clauses in the web.xml file would prevent abuse.

In the production version of WeatherCurrents.com, Tomcat is proxied behind Apache HTTPD. Rather than let Apache handle the PUT method, I wanted programmatic control over it.

So I built a servlet filter to handle HTTP PUT request from the network camera. Since filter chains are executed before any servlet (the DefaultServlet included), this would allow fine-grained control, and avoid having to use, modify or configure the DefaultServlet in any way.

The design of the filter takes into account the following:

  1. Only look at authenticated PUT method requests. Everything else passes through gracefully.
  2. To prevent abuse, use an upper limit on the size of the content payload. I chose 100,000 bytes arbitrarily. The actual images, in 640x480 format from the camera, are 35k or less.
  3. Recognizing that the system may want the uploaded picture in a specific place, I provided a way to configure a directory to place it. On Linux systems, this directory must have permissions for Tomcat to write in it.
  4. Create a role in Tomcat for the web camera. I called it "camera".
  5. Create a username and password for HTTP Basic authentication, and bake the authentication pattern into the web.xml file, again targeting only the PUT method.
  6. Make sure the content that we get is a JPEG image. For this, I took and modified some code originally developed for Apache Cocoon and released under the Apache 2.0 license.

Here's what the filter looks like. This filter is a simplified version of the actual one in use in the WeatherCurrents system:


public class PostNetworkCameraFilter extends HttpFilter
{
    // Sidestep DOS attacks with large files
    private static final int MAX_BYTES_LENGTH = 100000;
    private String basePath;


    @Override
    public void init(FilterConfig config) throws ServletException
    {   
        Properties props = FilterUtility.configToProperties(config);
        basePath = props.getProperty("base-directory",
                "/opt/apache-tomcat/webapps/static/cameras/");
    }   


    @Override
    public void doHttpFilter(HttpServletRequest request,
            HttpServletResponse response, FilterChain chain)
            throws IOException, ServletException
    {   
        Logger logger = Logger.getLogger(getClass());


        // Examine PUT requests only
        if (request.getMethod().equals("PUT"))
        {       
            // Check the length of the submission. No large files get through.
            if (request.getContentLength() > MAX_BYTES_LENGTH)
            {                                  response.setStatus(HttpServletResponse.SC_REQUEST_ENTITY_TOO_LARGE);
                logger.error("uri: " + request.getRequestURI() + " size "
                        + request.getContentLength()
                        + " is larger than maximum size " + MAX_BYTES_LENGTH);
            }           
            else        
            {           
                byte[] content = contentToBytes(request.getInputStream(),
                        request.getContentLength());
                // Check for a legal JPEG submission
                ImageProperties imageProps = ImageUtils
                        .getJpegProperties(new ByteArrayInputStream(content));
                if (imageProps == null)
                {               
                    logger.error("uri: " + request.getRequestURI()
                            + " is not a valid jpeg file");
                response.setStatus(HttpServletResponse.SC_UNSUPPORTED_MEDIA_TYPE);
                }
                else            
                {               
                    // Store the image where it can be accessed
                    logger.debug("uri: " + request.getRequestURI() + " size: "
                            + request.getContentLength()
                            + " image properties: " + imageProps);
                    storeImage(new ByteArrayInputStream(content), new File(
                            basePath, request.getServletPath()));
                                                        
                    // Other things we might do:
                    // 1. Check for an existing file and rename it
                    // 2. Notification for the rest of the system
                }
            }

        }
        else
        {
            // Pass through everything else
            chain.doFilter(request, response);
        }
    }

    private byte[] contentToBytes(ServletInputStream is, int length)
            throws IOException
    {
        try
        {
            ByteArrayOutputStream os = new ByteArrayOutputStream(length);
            try
            {
                byte[] buffer = new byte[4096];
                for (int n; (n = is.read(buffer)) != -1;)
                    os.write(buffer, 0, n);
            }
            finally
            {
                os.close();
            }
            return os.toByteArray();
        }
        finally
        {
            is.close();
        }
    }

    private void storeImage(InputStream is, File file) throws IOException,
            FileNotFoundException
    {
        try
        {
            file.getParentFile().mkdirs();
            OutputStream os = new FileOutputStream(file);
            try
            {
                byte[] buffer = new byte[4096];
                for (int n; (n = is.read(buffer)) != -1;)
                    os.write(buffer, 0, n);
            }
            finally
            {
                os.close();
            }
        }
        }
        finally
        {
            is.close();
        }
    }

}


Here's what needs to go in the web.xml file:


    <!-- For getting network camera images -->
    <filter>
        <filter-name>Network Camera</filter-name>
        <filter-class>org.toman.webcam.PostNetworkCameraFilter</filter-class>
        <init-param>
            <param-name>base-directory</param-name>
            <param-value>/opt/apache-tomcat/webapps/static/cameras/</param-value>
        </init-param>
    </filter>
    <filter-mapping>
        <filter-name>Network Camera</filter-name>
        <url-pattern>*.jpg</url-pattern>
    </filter-mapping>


    <!-- Make sure all requests to PUT network camera images are authenticated -->
    <security-constraint>
            <display-name>HTTP PUT Authentication</display-name>
                <web-resource-collection>
                    <web-resource-name>Cameras</web-resource-name>
            <url-pattern>*.jpg</url-pattern>
            <http-method>PUT</http-method>
        </web-resource-collection>
        <auth-constraint>
            <role-name>camera</role-name>
        </auth-constraint>
    </security-constraint>


    <security-role>
        <role-name>camera</role-name>
    </security-role>


    <login-config>
        <auth-method>BASIC</auth-method>
        <realm-name>Authentication Needed</realm-name>
    </login-config>


Note: the two pieces of code above are just snippets. The complete code is here, and you can use most of it under the terms of the FreeBSD license. Two classes are under the Apache 2.0 license. Gradle is used to build the source, but you can easily use Maven or Ant instead.

I haven't tried this in Jetty, or in Glashfish, but there's every reason to think it will work in those servlet containers the same way as Tomcat.