Optimization Techniques:

Optimization is the practice of finding a minimum or maximum value of a mathematical system under certain circumstances.  From a Business Process Management perspective, optimization is finding the best settings for your process in order to get optimal results.  An example of an optimization problem would be to find the best staffing levels on each of three separate, overlapping, shifts in order to maximize throughput while minimizing costs.

Imagine searching for the highest elevation in a large forested area, like a national park.  The rules are that you can use help from friends, as well as an altimeter and GPS to assess your elevation and coordinates only.  Imagine the area is fenced so your search area is clearly defined.  Unfortunately, the area is forested so a simple glance towards the horizon won’t offer much help.  The area of the forest is also quite large, so it would take an exceptional amount of time and effort to search the entire location.  This is the problem that an optimization engine is faced with when trying to optimize your business processes.

There are many mathematical methods of optimization, each suited for a particular type of problem.  Some systems, however, are too complex to find an optimal solution using standard mathematical techniques.  In order to optimize these more complex systems, researchers have developed creative ways of searching for an optimal solution in efficient ways.  These more creative search methods are called Heuristic techniques, and they are essentially “intelligent guessing” so that your search is more efficient than merely stumbling in the dark.  Below is a brief description of some of these methods:

Steepest Descent:

  The reason for the name of this method is that you are always moving in the direction that takes you downwards, or in our case upwards, the quickest, similar to the way water flows in the direction of least resistance towards sea level.  While searching in the forest you would simply walk towards the direction that is highest.  After several steps, re-assess your direction until you cannot find a direction that takes you to a higher position.  At this point you are at one of the highest spots in the forest.  The problem is that there could be another, higher peak on the opposite side of the forest.  The way around this is to get help from several friends that begin their search at random locations throughout the forest.  After all friends compare their results choose the highest location found.  Enlisting friends in this way is a generalization of Global Optimization methods which seeks to find the highest peak in an area with many peaks and valleys, rather than one single peak.  This method is usually used when the system being optimized has no uncertainty or randomness, and it can often be a slow and tedious process. 

Genetic Algorithm:

 The Genetic Algorithm searches for an optimal solution very similar to the way living organisms evolve with each generation.  The strongest solutions mate and produce (hopefully) stronger solutions, while the weaker solutions die off.  It is survival of the fittest. 

To adapt the Genetic Algorithm to the forest search, first choose a limited number of locations to search initially with the help of friends.  Afterwards, all friends compare their results and then combine them to form new search locations in the following way:  Take two locations and combine them to form a single new one by taking the longitude of the first and the latitude of the second.  Each time we choose two locations to combine we make sure at least one of them has a relatively high elevation.  After producing new locations the friends each take one of the new ones and assess that location for height.  Each time the friends meet and compare results a new ‘generation’ of locations is born. After several generations there will be a large pool of searched locations to draw from, so always try to choose stronger (higher) locations to combine.  After a pre-determined number of generations, or after a couple new generations produce no new improvements, pick the highest location from the entire pool.

Greedy Algorithm:

Rather than searching for the highest peak in a forest of varying grades, the Greedy Algorithm seeks to optimize discrete systems.  For our example, suppose there are a number of locations in the forest we wish to examine and compare, such as the locations obtained from the Genetic Algorithm described above.  The Greedy Algorithm would be used to optimize the order in which we visit these locations in an effort to minimize our walking, which is classically known as the “Traveling Salesman Problem.”  The Greedy Algorithm simply says that you always visit the site closest to your current location.  This method is described as “greedy” because you always choose the option that is currently easiest with no regard for your long-term future.  This is the main problem with the Greedy Algorithm: it often overlooks better solutions because of its near-sightedness.  Often it may be more optimal to your overall trip if you sometimes choose a farther destination in order to save steps later on.

Ant Colony Optimization

The Ant Search behaves very similar to the way an ant colony searches for food sources.  Pick a random location for your base camp and send your friends to walk away from the camp, each in different directions.  If you friends find a local high spot they mark it with a flag and return to the base camp, while leaving a trail of bread crumbs.  If they do not find a local high spot before they tire out they turn around and come back, then choose a new direction to search.  Eventually there will be several trails of bread crumbs, but unfortunately bread crumbs aren’t reliable in the forest so they might deteriorate.  If anyone happens across a trail they are directed to follow it as best they can, but if they loose the path its ok - just continue the search for high ground from that point on.  If someone should find a flag they return again with a trail of bread crumbs, strengthening the original trail.

 While this method doesn’t seem much more efficient than the Steepest Descent, it is very effective in dynamic systems that are always undergoing change.  If you imagine that the forest is on unstable ground so that new peaks and valleys are always forming then the Ant Search would quickly and efficiently identify current high spots.  As the high spots erode back to ground level the ants, who are always on the move, abandon the location for a new higher peak.  The Ant Search is best suited for shortest path problems such as the Traveling Salesman Problem.

There are many other interesting methods that are specially suited for specific types of problems, such as the Tabu Search which simulates the human brain, or the Simulated Anealing which simulates the way atoms achieve equilibrium under certain conditions.  The lesson is that if you are striving to achieve an optimal balance between competing factors, such as cost versus quality, a similar problem has probably already been solved by nature.

Obviously these descriptions are merely summaries of the concepts behind the optimization methods.  In practice, these optimization methods are quite technical and they are applied to problems that are significantly different from finding the highest spot in a forest.  What they all have in common is that they provide efficient ways of finding a needle in a haystack through intelligent guessing rather than exhaustive searching.