Artificial Intelligence as a decision-maker in logistics

Artificial intelligence has started to impact the logistics industry, along with the supply chain. Most examples we see in (un)supervised learning applications. In partnership with Bullit, we developed a tool that is able to learn how to connect individual shipments.

Bullit Digital helps companies digitizing their business processes according to the latest IT innovations. They develop custom apps, web applications and software for companies that want to lead the way. In addition, they help companies effectively deploy the latest IT innovations such as artificial intelligence and Big Data. Emons Cargo 2WIN offers Europe’s largest multi-customer double deck freight network for FTL shipments with units between 130 – 180 cm.

To explore the possibilities of artificial intelligence (AI) within the supply chain of freight, Bullit organized an AI workshop for Emons. The outcome of all possible ideas eventually led to a proof of concept:

“Can we use artificial intelligence to link the individual trips of tenders in such a way that Emons drives as few empty kilometres as possible?”

On average almost 20% of the trucks drive empty within Europe.  By cleverly connecting trips, Emons can cover valuable kilometres more efficiently and therefore more sustainably. Easier said than done, of course, since the possibilities are enormous. For the proof-of-concept, we used an existing dataset containing a list of shipments where freight carriers could quote on. We start on the journey with Built Digital and Deltago to discover the AI potential.

Our dataset contained about 1000 routes, including loading location, unloading location, frequency, distance in KM, and other known relevant specifications. First of all, we integrated all locations with Google Maps API to display several coordinates.

Because the possibilities for the tooling increase exponentially with each additional location, we have chosen to cluster some locations. This allows the tooling to be more efficient in identifying routes of interest.
In addition to the locations, the size of the circle indicates how many shipments depart from or arrive at the cluster. This provides an initial indication of the “better” fitting locations. Where the collection and delivery clusters are located together and the circles are comparable, Emons probably drives very little empty.

To create a clear view between collection and delivery clusters, we have combined all trips per cluster. This provides us more insight into the average distance between the clusters, the frequency of the trip and the number of trips. By estimating the distance between a delivery cluster and the nearest collection cluster, we gained more insight into the empty kilometres of a route.

For this tooling, we used reinforcement learning, a technique within machine learning that allows a tool to learn based on simulation. To simulate, three variables are important; the scope, the scoreboard and the tooling.

The scope or region where the tooling can move exists out of all pickup clusters. To make sure the tooling knows where he has been, it remembers the last 5 steps. This allows the tooling to make a well-considered decision based on past actions. This is important for the frequency of the trips.

The actions that the tooling can take in the scope are decisive to which delivery cluster he wants to drive. In addition, the tooling can also decide that the trip has ended, for example when there is no longer a suitable trip.

To train the tooling and make it more intelligent, we should tell him how well he did. Therefore we are using a scoreboard and assess the tooling on 3 criteria:

1. What is the distance of the chosen trip?
The tooling learns that there is a preference for longer trips.

2. To what extent does the frequency of trips match?
When the frequency between individual trips is in line, the chance of empty kilometres can be considered as lower.

3. What is the empty mileage ratio?
When a load has been delivered, the truck drives empty to the next pick-up location. By integrating this ratio, the tooling advice on the next preferred delivery location, including a close by loading possibility.

The tooling consists of the neural network, in which the knowledge is stored. By one-hot-encoding, we can ensure that the tooling recognizes “see” the playing field – and the sequence of the rides.

Then we use Advantage Actor-Critic (A2C) as a technique to train the tooling. This technique is very suitable for this case because it tries to predict the value of the scope and the possible advantage of any action taken. This allows the tooling to estimate the best next step.

When training the tooling, it starts at a random pick-up location, takes actions and receives scores. Based on these scores, the tooling continuously adjusts its neural network to make a better future prediction. After sufficient training and more and more simulations, the tooling is able to achieve better results. In just 10 minutes of training, the tooling will be able to turn the negative output into positive scores, getting closer and closer to the best outcome. Even when we add more requests for quotations, the tooling is still able to identify well-fitted lanes.

Below are the results of average scores achieved by the tooling. An interesting observation is that after about 4000 iterations the tooling continues to learn and optimize. The average score keeps increasing slowly.

To show that the tooling is capable of keep on making smart decisions, we asked it to tie trips together before and after training.  

Non trained
Using the tooling untrained, it will select randomly its actions. This frequently will lead to a selection of delivery clusters with non-existing orders. Logically this will result in a poor score.

AI training
When trained, the tooling is able to advise on interesting and successive trips. It will no longer suggest driving somewhere empty without any reason. In addition, it also matches frequencies and prefers longer journeys.

Deliverables
With this proof-of-concept, we have shown that reinforcement learning is a suitable technique for making faster and better decisions in logistics.

Themain results from this pilot are:
– In dozens of cases, AI can add value for Emons and its customers,
– Reinforcement learning has been successfully applied to support the operational planning of trips where empty kilometres are driven as little as possible,
– Determining the scoreboard for a combination of trips helps Emons evaluate and improve its own approach.

The article is based on an article by TKI Dinalog.