In this post we’ll go over setting up the popular
mitmproxy tool on an external host and configuring your Java programs to proxy traffic through it, allowing you to debug misbehaving HTTP clients and libraries.
Occasionally, you’ll be faced with a buggy Java program that is using HTTPS to communicate with an API. You could simply dump everything out using
-Djavax.net.debug=all, but sometimes this approach is cumbersome, as it only lets you view traffic, not replay it or modify it. This is where
mitmproxy comes in.
In this post, we’ll set up
mitmproxy on an external host and use it to intercept some HTTPS traffic from our example client. Let’s get started!
mitmproxy is a python program that transparently proxies any traffic sent to it. Through its console interface, you can inspect, capture and modify HTTP/HTTPS traffic flows as they are happening. Think of it like a step-through debugger, but for HTTP requests and responses.
The example client
We’ve created a simple java program for this blog post that will serve as the target application we wish to capture traffic from. This program (shown below) simply does a GET via HTTPS and prints out the certificate issuers for the
example_url system property, defaulting to
Here is what our example client looks like (imports and exception handling omitted for brevity):
If we run it with no arguments, the output looks like:
If you’d like to follow along, feel free to download or compile the example program.
Setting up the
Now that we have verified that our example program works as intended, we can continue setting up
mitmproxy on an external host.
Traditionally, most documentation and tutorials will have you install
mitmproxy on the same host you wish you capture traffic from. However, we prefer using a cheap disposable VPS instead, this way you avoid any issues around setting up a working python environment on your local machine.
For this example, we’ll be using the $5 VPS from our friends over at DigitalOcean. Select the latest Ubuntu version from their dropdown and login to the machine once it’s ready.
Since this package is available from Ubuntu’s repositories, we can just install it with
Since networks tend to block the default proxy port
8080, we’ll start our
mitmproxy server on port
If everything was installed correctly, it should bring up the
mitmproxy console interface, letting us know the server is running and accepting connections on port
(Make sure to leave this session open and running)
Configuring the Client
In order for
mitmproxy to successfully intercept and modify encrypted HTTPS traffic, Java needs to trust our proxy as a Certificate Authority. For the specifics on how this works in practice, see: Implementation Weakness of the Trusted Third Party Scheme and How mitmproxy works.
Viewing the generated certificates
When we started
mitmproxy above, it should have generated a self-signed certificate bundle inside of
~/.mitmproxy. If we inspect that directory, we should see that it generated a few certificates in various different formats.
Transferring the certificates to client
Since Java accepts PEM formatted certificates, we only need to copy the
mitmproxy-ca-cert.pem file to our client computer.
Import certificate into Java KeyStore using the
Before we can import our certificate into the Java KeyStore, we need find the
lib/security/cacerts location for our Java installation. If the
JAVA_HOME environment variable is set, this is located at
$JAVA_HOME/lib/security/cacerts. However, if this variable is not set, you’ll need to consult your system’s Java documentation for instructions.
keytool command, located under
$JAVA_HOME/bin/keytool, is a utility that manages keys and certificates for the Java keystore. To import our certificate we run the following command:
You will be prompted for your
sudo password first, then you will be prompted for the keystore password:
The default password for the Java keystore is:
Upon successfully entering the password you will be shown the
mitmproxy certificate and asked if you want to trust it:
yes and you should see:
Running our example
We’ll call our example program again, this time passing in our proxy settings as arguments. You can also export a
JAVA_OPTS environment variable containing the proxy configuration for situations where you can’t control the program’s arguments.
Note: it is important that the proxy configuration come before the
If everything was setup correctly, you should see that the certificate for
https://www.google.com is now issued by our own Certificate Authority,
Subsequently, in our
mitmproxy session, we should also see the intercepted flow on the UI:
enter on a selected flow lets you inspect it, hitting
tab lets you cycle through Request, Response and TCP details.
q to go back to the list of flows.
Feel free to run the client a few more times so you can see how the
mitmproxy UI handles multiple flows.
? for a list of things you can do to any selected flow.
A particularly useful one is
r, which replays the selected request. Very handy for server development.
This minimal example should serve as a playground to explore the concepts around debugging Java HTTP behavior using
mitmproxy. For bonus points, try intercepting some
sbt flows using
A full guide to using
mitmproxy is out of the scope of this post but you should definitely check out their comprehensive documentation for a thorough guide on all of the things
mitmproxy can do.