Using the “Virtual Proxy” Design Pattern in Python
You’ve likely heard the term “proxy” before, probably in the form “Proxy Server” used in the context of computer networking. A Proxy Server stands in for another resource, fielding requests on its behalf. That’s effectively what “proxy” means: “something that is used to represent something else”.
In that vein, a “Virtual Proxy” stands in for a resource that is expensive to create, acting as a surrogate before or while that expensive resource is created. Ever clicked on a webpage and been greeted by a loading bar or spinner? Both are examples of Virtual Proxies, allowing you to be greeted by something other than a blank screen while the webpage loads.
The expensive resource we’ll be creating a Virtual Proxy for is a Selenium Webdriver. For the uninitiated, this is an object you’d use to programmatically control a web browser. There aren’t many things that I’ve tinkered with that are more CPU and memory intensive! (Especially when that web browser is Chrome …)
Now, onto our contrived example.
(Note: in the following section bolded terms are objects used in the examples, italicized terms are used to refer to method calls)
Let’s say we have a large software project. Lots of objects and modules interacting with each other. One of these objects is a Scraper responsible for grabbing data from a target website. The Scraper does this by leveraging a Selenium Webdriver which it stores in its driver attribute:
The issue with this approach is that everytime we instantiate a Scraper it spins up a webdriver which is computationally expensive. We may not need the webdriver initially or at all. This is particularly the case when we’re integration testing different parts of the system that may require a Scraper but not leverage its functionality in that immediate context.
It’d be better if we waited to spin up a webdriver until when we knew we needed it, like when we issue a driver.get(“someurl.com”) command. To that end we could do something like this:
This is slightly better: We’ve deferred spinning up the webdriver to when we need it. The problem with this solution is its maintainability. If we add methods to the Scraper class that also issue driver.get(…) commands or if we have other objects that access the Scraper.driver attribute we need to remember to include some version of the same “if not self.driver…” logic everytime.
In fact, there is! This is where our Virtual Proxy comes to the rescue:
Let’s step through this. When we instantiate the Scraper we store a DriverProxy object in its driver attribute that stands in as a Virtual Proxy for the webdriver object. When we instantiate the DriverProxy we also pass in a reference to the Scraper so that the DriverProxy knows where to instantiate the more expensive resource (the webdriver). All this DriverProxy then does is wait for the Scraper to issue a driver.get(…) command. When this command is called, the DriverProxy spins up a webdriver and assigns it to the Scraper.driver attribute, implicitly removing the reference to itself. It also reissues the driver.get(…) command to this newly instantiated webdriver.
We now no longer run into the issue faced in the previous example! We can extend the functionality of the Scraper with methods that also issue driver.get(…) commands to our heart’s content without needing to remember any additional logic.
It’s important to note that Proxies should have the same interface as the objects they’re standing in for. If we end up accessing Scraper.driver via a method that is different than the driver.get(…) call we need to add that method to the DriverProxy with similar logic to what we saw in the DriverProxy.get(…) method. However, the upside is that all of these changes and logic are encapsulated into a single place, the DriverProxy object.
For further reading check out the excellent “Head First Design Patterns” book and this Python UML diagram of the Proxy Pattern.
Happy coding!
