0

Ok this is hard to explain and im not sure if this is the right place to ask it but.

Im 2 weeks in on my new work and I feel like our work can be a lot more easier if we dont rely on just Excel for our tasks. We deal with a lot of excel files. And these excel files are a lot. Per day we have 2000+ entries. So when we do monthly data, that excel file would be 2000x30 days. Imagine that.

Basically we just do matching with these excel files. All these excel files have different formats but they all have this unique mobile number (i work at a telco company) so we bash data with the excel files we have. Currently, we just use Access to bash data. But its not efficient. Access is REALLY SLOW.

So its like data mining. Lets say we are given this mobile number they want to investigate (we are in the fraud analysis team) so its my job to backtrack and check that phone number's usage. But our company has an inefficient way of handling their data. Like theres various excel files that arent centralized.

Anyway, any suggestions on how to deal with this? Are there alternative tools? etc. Since you guys know about database and i know its a rule that Excel shouldnt be used for database management. I do have some background with mysql and mssql but thats just basic commands.. Im not sure if it can be incorporated with my job.

(also, is there a word for what i do exactly? they call it data mining. We just match a mobile number and look for its corresponding data across different sources)

Improve this question
2
  • How about importing excel files into SQL server using SSIS. This can be automated, but you would need SQL server. If your database size is less than 10GB, you can use SQL express edition.
    – Kin Shah
    Commented Aug 6, 2015 at 12:37
  • Not sure if they even know what an SQL server is honestly. What do you think our database was? Its not a fancy kind of database and its not even centralized. Like we need to get various files from different tools. One is a text file that is 2gb in size where all mobile numbers and their balance our put in there since 2012. We match the excel file with this text file via Access. which is a tedious process.
    – user1551672
    Commented Aug 6, 2015 at 12:40

1 Answer 1

Reset to default
2

What you're describing is known as "Excel spreadsheet hell". Your scenario sounds like a disaster - constant firefighting, messy data... the list goes on.

I'm not sure what sort of data volumes you're dealing with. Oracle's solution to this is Apex. It's potentially a lot more than that, but take a look here.

If your volumes are not too large, you could use Oracle XE as your backing database for your systems. Apex only works with Oracle - hardly surprising for a tool which comes free with their database system.

You might ultimately want to upgrade to one of the other Oracle editions - but ultimately, managing data is not a trivial exercise and in order to improve your systems, some financial outlay may be required.

You could take one of the simpler systems, convert it to Apex and show management the potential of using a proper RDBMS to manage their data. It might require some midnight oil on your part, but if it works, it could be of benefit to your company and you.

All I would say, is that your system is untenable and will collapse under its own weight at some point in time. Either try and make it better or resign.

p.s. I hope you don't work for my telco! :-)

Improve this answer
2
  • I have no background in Oracle.. I was expecting for someone to just suggest something more basic like... SQL.. Cause usually its excel .. then access.. then SQL. I wont consider our company's data messy though. The data is just spread out (tho that means its messy but not really haha). Like the data is coming from different sources that's why it makes it complicated. Im sure we don't work at the same Telco since I don't live in the US :) but im working on a Telco that's pretty big in our country.
    – user1551672
    Commented Aug 7, 2015 at 0:41
  • 1
    To use SQL, you're going to have to put the data into an RDBMS. Oracle XE will take your data if it's less than 11GB (IIRC). SQL Server Express is similar. You could use any server - PostgreSQL, SQLite, Firebird... the list goes on. Basically, the problem is that properly organising data takes time, effort and possibly money. People become comfortable with the uncomfortable and tend to use what they know, i.e. Excel, no matter how great the inconvenience. Excel is good for presenting subsets of data that ideally come from a database. It is not a data management tool!
    – Vérace
    Commented Aug 7, 2015 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.