Categories
Development

Scrapy to get dynamic business directory data thru API

In this post I want to share on how one may scrape business directory data, real estate using Scrapy framework.

Categories
Development

PHP Curl POSTing JSON example

We share here the example of CURL POSTing JSON data to obtain an Octoparse API token.

<?php
$base_url = "https://openapi.octoparse.com";
$token_url = $base_url . '/token';

$post =[
    'username' => 'igorsavinkin',
	'password' => '<xxxxxx>', 
	'grant_type' => 'password' 
];

$payload = json_encode($post);

$headers = [
	'Content-Type: application/json' ,
	'Content-Length: ' . strlen($payload)
];

$timeout = 30;
$ch_upload = curl_init(); 
curl_setopt($ch_upload, CURLOPT_URL, $token_url);
if ($headers) { 
	curl_setopt($ch_upload, CURLOPT_HTTPHEADER, $headers);
} 
curl_setopt($ch_upload, CURLOPT_POST, true); 
curl_setopt($ch_upload, CURLOPT_POSTFIELDS, $payload /*http_build_query($post)*/ );
curl_setopt($ch_upload, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch_upload, CURLOPT_CONNECTTIMEOUT, $timeout);
$response = curl_exec($ch_upload);

if (curl_errno($ch_upload)) {
    echo 'Curl Error: ' . curl_error($ch);
}

curl_close($ch_upload); 
//echo 'Response length: ', strlen($response);
echo  $response ;
$fp = fopen('octoparse-api-token.json', 'w') ;
fwrite($fp, $response );
fclose($fp);