Group Group Group Group Group Group Group Group Group

Web scraping app optimizations

So I’ve been working on an app for quite some tim nowI. t’s actually my first real own iOS app ever, that I wan”to release on the App Stoe

he idea behind the app is quite simple;

It get whats being set what’s being served for lunch today at some local restaurants and displays it to the user by simply scraping it directlyfrom a number of websites .

Up until now the app retrieves what’s being served for lunch today on some local restaurant websites, for now it works like a charm but i’m having a hard time getting my head around that it has to be a more efficient and longterm way of doing this.

What I mean by long term is that if any of the web pages change, it could break the whole app.

One option I’ve been thinking about is to maybe have a PHP script that does the heavy lifting (retrieving, parsing,picking out the right information , but then I would probably have to pay for the expenses of a webhosting solution, which I would rather avoid

So all of this boils down to my question, what do you think I should do?

Alny suggestions is greatly appreciated

I think you are on the right thinking path with regards to having the heavy-lifting being done elsewhere. The reason why is UX (user experience) - let me explain: What happens if one of those sites is down? Or unreachable? Or they change the content and you can no longer parse it? The user will be immediately impacted.

If you do the heavy-lifting at your end, upload the results to some service, then the app hits your service, which you control. If something goes wrong with parsing, you will know and fix, but the app doesn’t stop working.

As for the costs, you will be surprised how low they are nowadays - some even free! Do some web-searching and you will find options.

Where should I store the file then?
Can I use Dropbox?

I do believe that Dropbox has some kind of public file sharing, I don’t know if I would personally would go that route. I’d rather setup standard web hosting. There are hundreds of solutions out there at very discounted rates and some even are free (Google: free hosting).

If you have some level of experience with web hosting, I would recommend AWS:

Is Firebase a good alternative?

Firebase is a fantastic framework, and it will definitely solve your issue - in a different way. You can store your data in its database and then pull it in.

Firebase is not a website, which was your original question.

But how and where do I populate the database?

I’m afraid that explaining Firebase usage would go beyond the scope of an Off-Topic Forum question. However, I can recommend some tutorials from the RayWenderlich.com website:

RW Pro:

Thank you for the uppish answer

I’m sorry, but what would I gain by storing it in Firebase?