Introduction
Design patterns are used to architect a software product. They are needed at higher- level as an abstract solution to implement the best coding practices for object oriented programming. The first-class citizens in python are most of the functions. which means you can pass them to other functions as arguments, return them from other functions as values, and store them in variables and data structures.
To lay it out plainly, design patterns are regular answers for basic issues when composing programming. What makes design patterns so useful is that they’re so generally applicable to a large number of problems, however you need to know how to apply them. You can learn more inside and out about these common design patterns (here)[https://en.wikipedia.org/wiki/Software_design_pattern].
In this article we will talk about design patterns that can be implemented while we work on our AI/ML software architecture. So, without further ado, let’s get into 3 great design patterns for data science workflows.
1. Dependency injection
In its most straightforward structure, dependency injection is the point at which you embed the thing you’re relying upon as an argument. Let’s for example consider you are working on a complex classification problem. However, you are not sure what will be the source of your training data. Your function doesn’t need to know how the data source class works, just that it does. Passing in the data source class instance as an argument makes it easier to maintain - you can use any kind of data source class that implements the same interface. Without using dependency injection, you’ll have a much harder time maintaining critical infrastructure like data source classes. Below is a sample that represents the example that we discussed above.
from dataclasses import dataclass
from datetime import timedelta
from typing import Optional, Any
@dataclass
class DataSouceInterface:
expiration: timedelta = timedelta(days=30)
location: Optional[str] = None
def get(self, key: str) -> Any:
raise NotImplementedError
def set(self, key: str, value: Any) -> None:
raise NotImplementedError
def is_valid(self, key) -> bool:
raise NotImplementedError
class DataSourceFilesystem(DataSouceInterface):
"""Data source using files"""
class DataSourceMemory(DataSouceInterface):
"""Data source using memory"""
class DataSourceDatabase(DataSouceInterface):
"""Data source using database"""
@dataclass
class HTTP:
_dataSource: DataSouceInterface
def _fetch(self, url):
return ...
def get(self, url):
if self._dataSource.is_valid(url):
# Use cached data
self._dataSource.get(url)
else:
data = self._fetch(url)
self._dataSource.set(url, data)
if __name__ == '__main__':
database = DataSourceDatabase(location='sqlite3:///tmp/http-cache.sqlite3')
filesystem = DataSourceFilesystem (location='/tmp/')
memory = DataSourceMemory (expiration=timedelta(hours=2))
http1 = HTTP(_dataSource =database)
http1.get('training_data')
http2 = HTTP(_dataSource =filesystem)
http2.get('training_data')
http3 = HTTP(_dataSource =memory)
http3.get('training_data')
Dependency injection design pattern has various plus points that are applicable to all languages, not just Python. One of the extraordinary advantages of utilizing this pattern is that your code is a lot simpler to compose tests for. Just write a mock class (i.e. a mock database class) and use that in your tests, rather than having to use code that runs HTTP requests and slows down tests.
Since dependency Injection doesn’t need any adjustment in code conduct, it very well may be applied to legacy code as a refactoring. The outcome is a client that is freer and that is simpler to unit test in disengagement utilizing stubs or counterfeit items that recreate different articles not under test. This simplicity of testing is frequently the primary advantage saw when utilizing this pattern.
Dependency Injection can be utilized to externalize a framework’s arrangement subtleties into design records permitting the framework to be reconfigured without recompilation (modifying). Separate arrangements can be composed for various circumstances that require various executions of segments. This incorporates, however, isn’t restricted to, testing. The other extraordinary advantage of dependency Injection configuration design in your AI/ML product is the decrease of standard code in the application objects. Since all work to introduce or set up dependencies is dealt with by a supplier component. Dependency injection permits the client to eliminate all information on a solid execution that it needs to utilize. This confines the customer from the effect of configuration changes and imperfections. It advances reusability, testability and viability.
2. The Decorator Pattern
Decorators in Python are utilized to expand the usefulness of a callable object without altering its structure. Essentially, decorator capacities wrap another capacity to upgrade or adjust its conduct. How about we compose a Python 3 code that contains instances of decorator usage:
def decorator_func_logger(target_func):
def wrapper_func():
print("Log Dara before processing", target_func.__name__)
target_func()
print("Log Data after processing", target_func.__name__)
return wrapper_func
@decorator_func_logger
def pipeline_component():
print('ML pipeline component')
pipeline_component ()
Output :
run python DecoratorsExample.py
(Log Data before processing ', 'target')
ML pipeline component
(Log Data after processing ', 'target')
Decorators define reusable code blocks that you can apply to a callable object (functions, methods, classes, objects) to modify or extend its behaviour without modifying the object itself. Consider that you have many functions in your script performing many different tasks and you need to add specific behaviour to all of your functions. It’s not a good coding practice to add the same code block in each function where you require that functionality. You can simply do this by decorating your functions instead. For example you need to audit the cleaning process of your training data as it passes through your pipeline. You can decorate all the functions in your pipeline with a generic function that logs the data as it passes through it.
3. The factory pattern
Instead of creating a direct object through init() work we can do it with design patterns that manage the formation of objects. In the factory design patterns, it is the job of factory function to produce a different object for all input parameters based on the request generated by the client code without the need to know the origin or the source of the class of the object generation. The basic idea of driving the factory design is to disentangle object creation. To follow the objects that were made from the class, a focal capacity would be significant than the client making the objects through the launch of the immediate class. Thus, the factory design pattern helps to improve the code and decrease the impression of the code by killing the multifaceted nature by dechaining the code from the object that creates the code and the object that devours the code. Consider the below example where a factory design pattern is implemented for language translation use cae.
# Python Code for factory method
class FrenchLocalizer:
""" it simply returns the french version """
def __init__(self):
self.translations = {"car": "voiture", "bike": "bicyclette",
"cycle":"cyclette"}
def localize(self, message):
"""change the message using translations"""
return self.translations.get(msg, msg)
class SpanishLocalizer:
"""it simply returns the spanish version"""
def __init__(self):
self.translations = {"car": "coche", "bike": "bicicleta",
"cycle":"ciclo"}
def localize(self, msg):
"""change the message using translations"""
return self.translations.get(msg, msg)
class EnglishLocalizer:
"""Simply return the same message"""
def localize(self, msg):
return msg
def Factory(language ="English"):
"""Factory Method"""
localizers = {
"French": FrenchLocalizer,
"English": EnglishLocalizer,
"Spanish": SpanishLocalizer,
}
return localizers[language]()
if __name__ == "__main__":
f = Factory("French")
e = Factory("English")
s = Factory("Spanish")
message = ["car", "bike", "cycle"]
for msg in message:
print(f.localize(msg))
print(e.localize(msg))
print(s.localize(msg))
Conclusion
In this article, I’ve demonstrated three different ways to utilize design patterns as an AI/ML engineer for more maintainable, viable code. At the point when you use a design pattern in AI/ML, your code quality goes up, your support is simpler, and your outcomes are simpler to repeat and share.
