Feed Scraper

Feed scraper

Do you want to take advantage of what Dotidot has to offer, but you're missing the data source? With Feed scraper, you can easily create your own feed using sitemap


In the menu, open the data source tab and use the + button to select the Create new data source option. You will see a list of available data sources, this time you will select Scraper.


Select the name of your future data source and enter the URL of your sitemap you want to scrape. You can also filter the pages and subpages you want to scrape. You can refine your selection using conditions. On the right side you have a preview of what will be scraped and an example of a random product.

In the Scraping tab you can choose html tags that will not be scraped. In case your site needs Javascript to render the content, turn on the Use browser to load content button. Use this option only as a last resort.

Now you can choose to get even more data, and for that you can either use CSS selectors and select information from your site that will then be loaded as attributes in your new data source. Or use our Scraper wizard, which allows you to select data in a visual way. You can read more here.

Then you can choose, what will Dotidot do if the input data source contains 0 items. 

Once you have created the scrape feed, you will be redirected back to the data feed menu where the new feed will appear after synchronization. Please be patient, the Feed scraper takes longer to load than a regular XML feed.



Was this article helpful?
Didn’t find what you were looking for?
Ask a question