Introducing mPACT® API Feeds
We continue to improve our mPACT® API and add important features to it. Our most recent addition is the ability to create feeds. You can create a feed with keywords or complex Boolean query and you will get entire set of results from our mPACT® database – in batches, if result set is immense. On subsequent calls you will only get new items that came into our system since your last call. This way you create an ongoing feed that always has fresh new posts in it. This feature is very important in tracking Twitter, Facebook and other social media sites for the topic that interests you.
Now, let’s get into details. When you have a query that yields many results, a limitation of 500 items it not for you. You would want to get all items, but what if there is a million results – or more? One response will not work because it will contain many megabytes of data for your application to digest, so you need to take your data in a batch format. This is exactly why we created this new feed feature. You give us request — again either us a set of keywords or as a complex Boolean query – we do the search and return items in several subsequent responses that contain amount of data that works best for you. Each response will have items that were not returned before (no duplicates, in other words) and we’ll keep sending batches of the size that works best for you until we return all the items that match your request. You don’t need to bother calculating anything, you just set up your application to keep sending the subsequent requests and take care of the rest.
Here’s how it works: the new method for this is called /feed/socialmedia. First, you do a regular request to initiate feed creation:
/feed/socialmedia?searchKeywords=Apple,IPhone&FeedID=0&ClientKey=<yourAPIkey>
We create a unique feed for this request and return it to you in response together with first batch of data. Part of the response will contain all the information that you need for subsequent calls:
<FeedID>1111-2222-3333-4444</FeedID>
<TotalNumItems>10000</TotalNumItems>
<BatchNumItems>500</BatchNumItems>
The most important here is FeedID, as you will need it for your subsequent calls. TotalNumItems and BatchNumItems can be used to inform your user about amount of data he will receive, but you don’t need to store it or use it in any of your subsequent calls. Instead, your next calls should only be done with the FeedID, all other parameters (except ClientKey) will be ignored, they are defined by FeedID that you use:
/feed/socialmeida?FeedID=1111-2222-3333-4444&ClientKey=<yourAPIkey>
The response will be similar to what you got on the first call:
<FeedID>1111-2222-3333-4444</FeedID>
<TotalNumItems>9532</TotalNumItems>
<BatchNumItems>500</BatchNumItems>
Here, you can see that total number of items is higher than 9500 (10000 – 500 from the first call). This illustrates that between subsequent calls our system will process new items that match your request and they will be taken into account.
You need to continue submitting the request until you get empty set of results. This means that you have taken everything from our system that matched your request. However, if you continue doing the consequent requests with this FeedID you will be getting new items – the ones that have entered our mPACT® database since your last query – every 10 minutes. If amount of these new items is more than 500 you will be automatically getting them in batches as described above. To reset the feed, you just need to make a request with FeedID equal to 0. This will restart the process from the beginning.
This is a very simple and neat way of creating endless streams of social media data that your users and applications can benefit from. These new feeds aren’t just limited to social media. You can also create feeds for articles (from mainstream publications) and blogs in the same manner using the following methods:
/feed/article
/feed/blog
These feeds work exactly as it is described above for social media. Give it a try and let us know if we can help!
