As you know, huge social networks are very useful instruments to improve business, especially IT-business. Developers, designers, CEO, HR- and Product-managers share some useful information, looking for useful acquaintances, business partners and co-workers. But how does one automatize the process of searching and attracting new people to your resource? With Phantombuster it’s not a problem at all. In our today’s article we will consider how to use the Phantombuster APIs in different areas. Let’s start!
A few words about Phantombuster
Phantombuster’s team has created APIs for various social medias.
The huge advantage of Phantombuster is that anyone can use it. You don’t need to know how to code, all you need is to follow the step-by-step guide. Create an account, and you will get a 14-day free trial to test this tool for your needs.
For what is it used? For example, extend LinkedIn network. With Phantombuster you can send the invitation to over a thousand people per day (you can send e-mails to the located people in tandem with https://hunter.io). Another usage is that developers can create their own scrapers based on Phantombuster.
Moreover, you can use API to extend your audience in Instagram, Slack, GitHub, etc. You can get acquainted with the full list here.
LinkedIn Search Export: Basic Level
To introduce to you how Phantombuster’s APIs work, we will test one of the ordinary APIs called LinkedIn Search Export. It extracts info from any type of developers’ profiles to a .csv file.
To use this API we need to have two things: a Phantombuster’s account (it can be free with 14-day-trial activated) and a LinkedIn profile (with some contacts).
Once we are prepared, we can start:
- Log in to your LinkedIn profile. Press F12, find the tab Application -> Cookies, first link (https://www.linkedin…) and find there cookie li_at
Copy this cookie to the API form.
- To finish configurations, we need to set some more values:
- LinkedIn search terms – the search query. For example, we need to find developers with scrape skill. Fortunately, LinkedIn allow us to use advanced search. This is our query: skills: scraping AND headline:developer
- Check the Search in your 2nd degree connections and Search in your 3rd+ degree connections boxes (or you can check all boxes, it’s up to you).
- Let it be the default value in Number of pages/posts to go through in the search results –
- Name of resulting CSV file – let’s name it.
- And we are ready to test it. Click the Launch button and let’s see the results.
Beside the csv file, we get data in JSON format and HTTPS API endpoint under the main console. This is how endpoint looks:
curl https://phantombuster.com/api/v1/agent/agent-id/launch --data "{\"output\":\"first-result-object\",\"argument\":{\"sessionCookie\":\" your-linkedIn-cookie",\"search\":\"skills: scraping AND headline:developer\",\"circles\":{\"first\":false,\"second\":true,\"third\":true},\"category\":\"People\",\"numberOfPage\":5,\"numberOfLinesPerLaunch\":10,\"csvName\":\"scrapers\",\"onlyGetFirstResult\":false,\"removeDuplicate\":true}}" --header "Content-Type: application/json" --header "X-Phantombuster-Key-1: your-phantombuster-key"
In the curl endpoint replace the following with your own private data:
agent-id your-linkedIn-cookie your-phantombuster-key
Timing and result characteristics
- Search duration time – 33 seconds.
- Search results – 50 founded developers that are interested in scraping. LinkedIn allows you to scrape information from about 2500 persons.
- OK, we’ve got the needed data, what now? One can send invitations, business offers to the people on this list. What to do if you have only a few contacts? You can use LinkedIn Network Buster to extend your audience, for example.
Creating custom API: Advanced Level
Now we will step to more complicated and engaging things – our own API creation with Phanombuster. As Phantombuster is based on NodeJS, we will use it as the template for our scraper. Go to My APIs page and create a new agent there.
The Phantombuster will insert the code for scraping the hacker news – https://phantombuster.com/script/1391
This code is like a template for your future scrapers. Launch it and you will get the list of latest news from hacker’s world.
Let’s create our own scraper, just for fun. It will be an easy node.js scraper to parse book titles and prices from books.toscrape.com . We modify the existing code to make it the following:
// Phantombuster configuration {
'phantombuster command: node'
// }
const exitWithError = (err) => {
console.log(`Error: ${err}`)
process.exit(1)
}
const needle = require('needle')
needle.get('http://books.toscrape.com/', (err, res, body) => {
if (err) {
exitWithError(err)
}
const $ = cheerio.load(body);
const data = [];
$('article').each((i, el) => {
data.push({
title: $(el).children('h3').children('a').attr('title'),
price: $(el).children('.product_price').children('.price_color').text()
})
})
buster.setResultObject(data)
});
As you can see, here we use three entities:
- Cheerio – the package for parsing HTML (as you can see, it’s similar to JQuery).
- Buster – our connection to Phantombuster. It allows us to use all its features.
- Needle – yes, we can use the request module instead of needle. But for some reason it’s more convenient.
Run the code in a console. And we’ve got the resulting object:
[
{
"title": "A Light in the Attic",
"price": "£51.77"
},
{
"title": "Tipping the Velvet",
"price": "£53.74"
},
{
"title": "Soumission",
"price": "£50.10"
},
….
]
It took only 3 seconds to scrape 50 books for our scraper.
And below is the HTTPS API endpoint. Use it to get access to your API agent from any place:
curl https://phantombuster.com/api/v1/agent/the-api-id/launch –data “{\”output\”:\”first-result-object\”,\”argument\”:{}}” –header “Content-Type: application/json” –header “X-Phantombuster-Key-1: your-phantombuster-key“
In this command you can see important data: the agent’s API id and Phantombuster’s key. It’s personal data, be careful with it.
cURL usage is not the only way to address to a needed API agent. In the documentation you can see examples of how to address to API by Node, Ruby or Python. Also, Phantombuster’s documentation allows us to run the code right there! All you need is the above-mentioned agent id and Phantombuster’s key.
In the same way you can manage your account or launch scripts.
Afterword
That’s all for this article, but not for Phantombuster! Upgrade to the gained info and use it to extend your audience in needed social medias. Hope this article will be a great and new start for your business.