Web Scraping Software

CharlesProxy HTTP Sniffer Review

The Charlesproxy website sniffer is the subject of this post. This sniffing/monitoring application works with Windows, Mac and Linux OS. It rather differs from other web traffic sniffing tools.

HTTP Proxy implementation

The Charles sniffer works by the principle of a proxy server. If I want to access a webpage and sniff its server headers and time delays, the Charles HTTP proxy receives requests from my web browser and then makes the request to the target server on my behalf. The result is returned again through the proxy. Charles documents each request and response between my browser and the target server.

The Charles sniffer automatically configures the proxy settings, so that the applications will immediately start using Charles. For detailed settings refer to here.

In the following picture there is the summary of several ajax requests (XMLHttpRequests) done by the browser:

Features of the HTTP sniffer

  • Catches headers
  • Catches the contents of requests and responses
  • Displays cookies
  • Supports HTTPS/SSL. Read here for details.

Additional features

  • Saving session.
  • Bandwidth throttling. Go to Proxy -> Throttling
  • Download statistics
  • Configurable. Go to Edit -> Preferences
  • Filtering results
  • Repeating request. Go to Tools -> Repeat 
  • Auto-save
  • Editing requests.

Some more features

These features pertain to the Charles proxy nature. The full description of these you can read here.

  • Set No-Cache
  • Block Cookie
  • Map Remote tool
  • Map local tool (maps local files as if they were part of a remote website)
  • Rewrite (rules that modify requests and responses as they pass through proxy)
  • Black list for domains
  • DNS spoofing (useful when testing virtual hosting immediately)
  • Mirroring
  • Repeat advanced tool for sending a request by the number of iterations and concurrency. Good for Load Testing.

AJAX debugging feature

The tool allows catching the actual XML sent by the client to the server. That works well when we want to debug Ajax requests (XMLHttpRequests).


The proxy based sniffing tool, Charles, is a good multi-featured application for professional use. It works well for retrieving all the HTTP traffic info, providing advanced features such as requests editing and re-sending, DNS spoofing and more. As far as the disadvantages of the sniffer, I might mention the lack of timeline, and work interruptions for the trial version.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.