Recently we discovered a highly protected site — govets.com. Since the number of target brand items of the site was not big (under 3K), I decided to get target data using the handy tools for a fast manual scrape.
Tag: JSON
Categories
PHP Curl POSTing JSON example
<?php $base_url = "https://openapi.octoparse.com"; $token_url = $base_url . '/token'; $post =[ 'username' => 'igorsavinkin', 'password' => '<xxxxxx>', 'grant_type' => 'password' ]; $payload = json_encode($post); $headers = [ 'Content-Type: application/json' , 'Content-Length: ' . strlen($payload) ]; $timeout = 30; $ch_upload = curl_init(); curl_setopt($ch_upload, CURLOPT_URL, $token_url); if ($headers) { curl_setopt($ch_upload, CURLOPT_HTTPHEADER, $headers); } curl_setopt($ch_upload, CURLOPT_POST, true); curl_setopt($ch_upload, CURLOPT_POSTFIELDS, $payload /*http_build_query($post)*/ ); curl_setopt($ch_upload, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch_upload, CURLOPT_CONNECTTIMEOUT, $timeout); $response = curl_exec($ch_upload); if (curl_errno($ch_upload)) { echo 'Curl Error: ' . curl_error($ch); } curl_close($ch_upload); //echo 'Response length: ', strlen($response); echo $response ; $fp = fopen('octoparse-api-token.json', 'w') ; fwrite($fp, $response ); fclose($fp);
Personally, I prefer using online tools for performing quick manipulation on different data formats like JSON, XML, CSV and so on. They’re platform independent and always within reach of my hand (since I mainly work in a browser). After we published an article about 7 best JSON viewers, I was told about Knowledge Walls, a similar service containing many tools for text data manipulation.
Recently, while surfing the web I stumbled upon an simple web scraping service named Web Scrape Master. It is a kind of RESTful web service that extracts data from a specified web site and returns it to you in JSON format.
Categories
7+ Best JSON Viewers
In this post we share on json viewers both as online tools and as plugins for browsers and Notepad++ editor.