What are some methods to analyse large sets of data in effective and efficient ways?

I have large amounts of data for clients of a transport provider (think similar to a taxi) in and around New York City. The kind of information I have is:

What area they are travelling to and from. How much stuff they are carrying (bag charges etc). Time taken to get from A to B. How much they paid for transport. etc.

So a typical line might look like:

Company XYZ | Taxi | Staten Island | Central Park | $34.90 | 1 Person | 0 Bags Obviously it differs from company to company but generally I have this type of information. I want a way to compare this information to what my company can offer and see if it is possible to offer clients savings on specific routes. We price on things like where you are going from and to (we have a map of the city + surrounding areas and it is split into zones), how many people, bags etc. And we offer flat rates so the example line might cost$30 for instance.

I am not looking for an out and out answer but instead am looking for some different methodologies I can investigate to see how best to analyse such data.

At the moment I am manually pricing each line for each client.

This is very very time consuming and in fact doesn't seem efficient as well because some client might use the same route 100's of times over the year so I am just repeating the work there.

Couple this with this with the many companies I am wanting to perform analysis with their data and I am looking at thousands upon thousands of lines.

I know this is a long shot but does anyone have any suggestions as to good processes to deal with such circumstances?

My thoughts are as follows:

1) I am likely going to have to sacrifice some accuracy to improve the speed of results.

2) I may have to only work with the top x% of lines by spend.

I am really looking for something I can research and wondered if anyone has tackled similar issues in the past?

• Do you have the same information from your own company? What you could do is train a model predicting price on data from your company, apply it to the data from other companies, and see where you can offer less. You might not even need a model—you could just get the average price for every combination and compare them. It just depends on the sparsity and complexity of your data. – Mark White Nov 21 '18 at 23:19