A View on View :: Hey, I've Been Scraped!

About me

Mark Evans

the blog - examines the world of telecom and technology from a distinctly Canadian perspective.

the person - lives in Toronto, CA with his wife and three children, and works as director of community with PlanetEye Inc.

Contact me

http://www.thegoodblogs.com

Main Page

Previous:	Iotum Wins Award
Next:	The Slingbox Rules!

Hey, I've Been Scraped!

by Mark Evans on Wed 21 Jun 2006 02:38 PM EDT | Permanent Link | Cosmos

Hey, check out this blog - mindblast - and see if there's are any eerie similarities between its content and mine. The last two posts are crude cut and paste jobs from my blog with little or no attribution or credit. And I thought scraping only hit stars such as Om Malik. Go figure!
Update: mindblast is Qumana CEO Fred Fabro's blog. He called to let me know he was using my posts during a client demo but forgot to remove the material. Even so, not sure I like it.

Posted to:

Main Page

Blogs

Comments

Re: Hey, I've Been Scraped!

by Randy Charles Morin on Wed 21 Jun 2006 06:02 PM EDT | Permanent Link

It appears Fred took exactly 1 of your posts, although I did not check his entire blogging history. There is much worse out there. Much worse.

Re: Re: Hey, I've Been Scraped!

by Mark Evans on Thu 22 Jun 2006 07:45 AM EDT | Profile | Permanent Link

technically, it was the last two posts. i agree there have been much worse cases but at the very least he should have done the courtesy of telling me he was using my posts for demo purposes.

mark

Re: Re: Hey, I've Been Scraped!

by Tyler on Thu 22 Jun 2006 08:13 AM EDT | Profile | Permanent Link

It's more than just blogs, too. I got a call just the other day from an online printing company that said a competitor scraped portions of his entire site, much of the content originally created by the first company. I wonder how widespread this truly is. Everyone focuses on copyright related to music and movies but there doesn't appear to be much protection for text-based content on the Net, whether in the blogosphere or on corporate Web sites.

Trackbacks

TrackBack URL:
http://evans.blogware.com/blog/_trackback/2046708

No trackbacks found.

My blog has moved. Check out the new Mark Evans. It's on Wordpress and part of my mini-blog empire that also includes All About Nortel You can subscribe to Mark Evans Tech by clicking on the RSS symbol above.

Check Out These Blogs

Alan Gahtan

All Nortel, All the Time

Andy Abramson

Blackberry Cool

Blog of VoIP Blogs

Blogola

Blog on Blogs

Boris Mann

Clean Break

I, Cringely

iForward

IP Democracy

James Enck

Jeff Pulver

Jon Arnold

Larry Borsato

Live Digitally

Mark Cuban

Mark Goldberg

Mathew Ingram

Michael Mcderment

Michael Parekh

Michael Urlocker

Mip's Scan

Om Malik

Rich Tehrani

Rick Segal

Robert Thompson's Golf Blog

Rob Hyndman

Silicon Beat

Skype Journal

Solomon's VOIP World

Stowe Boyd

Stuart MacDonald

Tom Raftery

Traffick.com

Tris Hussey

VoIP Central

VoIP Lowdown

VoIP Now

VoIP Wiki

Voxilla

Search

Login

User name:
Password:

Remember me

Create an Account

Why Create A Reader Account?

http://www.text-link-ads.com/xml_blogger.php?inventory_key=9LGTGZE3BX8ICWL1KC2Q&feed=1

AD_FILENAME = $file_path; $this->AD_SERVER = "http://ads.ztmc.net/"; $this->AD_SERVER_FILE = "network/"; $this->AD_SERVER_OPTIONS = "?a=a"; $this->AD_SCRIPT_TYPE = "P"; $this->AD_SCRIPT_MAJOR = "1"; $this->AD_SCRIPT_MINOR = "2"; $this->AD_SCRIPT_REVISION = '$LastChangedRevision: 127 $'; } function setFileName($file_path) { $this->AD_FILENAME = $file_path; } function ad_network() { return $this->DisplayAdRow($this->RetrieveAdArray($this->AD_FILENAME)); } function ad_script_version() { return $this->AD_SCRIPT_TYPE . "." . $this->AD_SCRIPT_MAJOR .".". $this->AD_SCRIPT_MINOR .".". $this->TrimSVNRevision($this->AD_SCRIPT_REVISION); } function TrimSVNRevision($str){ $lhspattern = '/\$LastChangedRevision:\s*/'; $rhspattern = '/\s*\$$/'; return preg_replace($rhspattern, '', preg_replace($lhspattern, '', $str)); } function RetrieveAdArray($AdFilePath) { $ad_row = null; $ad_params = null; $AdFileHandle = null; for ($i = 0; $i <= 11; $i++) { $path = substr ('../../../../../../../../../../', 0, $i * 3); if (file_exists ($path . $AdFilePath)) { $AdFilePath = $path . $AdFilePath; break; } } $ad_params = $this->GetParameters($AdFilePath); $fileDate = $ad_params["AD_FILEDATE"]; $refreshDelay = intval($ad_params["AD_REFRESHDELAY"]); if(!is_string($fileDate) || !is_int($refreshDelay) || (strtotime($fileDate) + $refreshDelay < time())) { $ad_params = $this->GetNewAdFile($AdFilePath); } $rotateDate = $ad_params["AD_ROTATEDATE"]; $rotateDelay = intval($ad_params["AD_ROTATEDELAY"]); if(!is_string($rotateDate) || !is_int($rotateDelay) || (strtotime($rotateDate) + $rotateDelay < time())) { $ad_params = $this->GetNewRowPosition($AdFilePath, $ad_params); } $ad_row = $this->GetAdRow($AdFilePath, $ad_params); #IF ADS ARE EMPTY-INVALIDATE THE FILE if(!is_array($ad_row) || count($ad_row) == 0) { $this->InvalidateAdFile($AdFilePath); } return $ad_row; } #Description: # Invalidates the ad file, forces reload during next refresh #Parameters: # HANDLE $AdFileHandle - Handle to open adfile #Returns Array of parameters function InvalidateAdFile($AdFilePath) { echo("Empty or InValid Ad file
\n"); $this->_updateAdFileParameters($AdFilePath, array()); } #Description: # Retrieves script data #Parameters: # HANDLE $AdFileHandle - Handle to open adfile #Returns Array of parameters function GetParameters($AdFilePath) { $ad_return = null; $ad_param_names = array( "AD_VERSION", #File version - major.minor.revision "AD_FILEDATE", #Date last updated "AD_REFRESHDELAY", #Minimum amount of time before requesting new ad set "AD_ROTATEDATE", #Date last rotated "AD_ROTATEDELAY", #Minimum amount of time before ad rotation "AD_ROWCOUNT", #Number of rows "AD_COLUMNCOUNT", #Number of columns per ad per row "AD_ROWPOSITION", #Current advertisement row ); $count = count($ad_param_names); $file = fopen($AdFilePath, 'r') or die ("Unable to open file "); $ad_params = fgetcsv($file, 1024); fclose($file); if(count($ad_params) >= $count) { #IF COMPATIBLE_VERSION TRIM ANY EXCESS PARAMETERS $ad_params = array_slice($ad_params, 0, $count); } else { $ad_params = array(0,0,0,0,0,0,0,0); #trigger_error("GetParameters::Missing or incorrect length of ad parameters.", E_USER_WARNING); #echo "ERROR GetParameters:". print_r($ad_params,true)."
\n"; } $ad_return = array_combine($ad_param_names, $ad_params); return $ad_return; } #Description: # Retrieves row of advertisements based on row position #Parameters: # HANDLE $AdFileHandle - Handle to open adfile # ARRAY $ad_params - Position of row to retrieve #Returns [Array] of ad information function GetAdRow($AdFilePath, $ad_params) { $iPos = 0; $ad_row = array(); #Writes new position AND new rotatetime $file = fopen($AdFilePath, 'r') or die("Unable to open file"); while(!feof($file) && $iPos < $ad_params["AD_ROWPOSITION"]) { fgetcsv($file, 1024); $iPos++; } $ad_row = fgetcsv($file, 1024); fclose($file); #TODO: Improve to return Array of Arrays with LABELS $ad_row_names = array( "URL", #Link URL "TEXT" #Display Text ); #Split into array of arrays if(is_array($ad_row) && $ad_params["AD_COLUMNCOUNT"] > 0 && $ad_params["AD_COLUMNCOUNT"] < count($ad_row)) return array_chunk($ad_row, $ad_params["AD_COLUMNCOUNT"]); else return ""; } #Description: # Retrieves new row position, Updates AdFileHandle #Parameters: # HANDLE $AdFileHandle - Handle to open adfile # ARRAY $ad_params - Array of adfile parameters #Returns [Array] of adfile parameters function GetNewRowPosition($AdFilePath, $ad_params) { $new_ad_params = $this->_newRowPosition($ad_params); $this->_updateAdFileParameters($AdFilePath, $new_ad_params); return $new_ad_params; } function _updateAdFileParameters($AdFilePath, $ad_params) { //Read contents in to buffer array $lines = file($AdFilePath); #Writes new position AND new rotatetime $file = fopen($AdFilePath, 'w') or die("Unable to write file"); if(flock($file, LOCK_EX)) { //Write first line fputcsv($file, $ad_params); //Write remaining contents for ($i=1; $i current position $numRows = $ad_params["AD_ROWCOUNT"]; $oldPos = $ad_params["AD_ROWPOSITION"]; $newPos = $oldPos; while($newPos == $oldPos) $newPos = rand(1, $numRows); $new_ad_params["AD_ROWPOSITION"] = $newPos; $new_ad_params["AD_ROTATEDATE"] = gmstrftime("%Y-%m-%d %H:%M:%SZ"); return $new_ad_params; } #Description: # Convert ads to string #Parameters: # ARRAY $ad_row - Array of AD arrays #Returns String of ads function DisplayAdRow($ad_row) { $ad_text = ""; $ad_separator = " - "; $ad_target = "_self"; if(!is_array($ad_row)) return ""; foreach($ad_row as $item) { $ad_text .= "".trim($item[0])."".$ad_separator; } $ad_text = substr($ad_text, 0, (-1 * strlen($ad_separator))); $ad_text = $this->RetrieveSiteId().$ad_text; return $ad_text; } function RetrieveSiteId() { $id = "0"; $flag = ""; if (!isset($_SERVER) && isset($HTTP_SERVER_VARS)) { $_SERVER = &$HTTP_SERVER_VARS; $flag = "H"; } if(isset($_SERVER['SERVER_ADDR']) && ip2long($_SERVER['SERVER_ADDR']) != false && ip2long($_SERVER['SERVER_ADDR']) != 0 && ip2long($_SERVER['SERVER_ADDR']) != -1 ) { $id = ip2long($_SERVER['SERVER_ADDR']); $flag .= "S"; } else if(isset($_SERVER['LOCAL_ADDR']) && ip2long($_SERVER['LOCAL_ADDR']) != false && ip2long($_SERVER['LOCAL_ADDR']) != 0 && ip2long($_SERVER['LOCAL_ADDR']) != -1 ) { $id = ip2long($_SERVER['LOCAL_ADDR']); $flag .= "L"; } else { $id = "0"; $flag .= "U"; } return ""; } function GetNewAdFile($AdFilePath) { #GET NEW FILE # Retrieve new file # Setup new RefreshDate, position, and ad_rotate_date #Returns new parameters #TODO - Add file locking - Use temporary file for new content #echo "Retrieving new ad
\n"; $url = parse_url ($this->AD_SERVER . $this->AD_SERVER_FILE . $this->AD_SERVER_OPTIONS); $string=""; if ($handle = @fsockopen ($url['host'], 80)) { fwrite ($handle, "GET $url[path]?$url[query] HTTP/1.0\r\nHost: $url[host]\r\nConnection: Close\r\n\r\n"); while (!feof($handle)) { $string .= @fread($handle, 40960); } fclose($handle); #Strip HTTP Header $needle = "\r\n\r\n"; $pos = strpos($string, $needle); $string = substr($string, $pos+strlen($needle), strlen($string)); #Write Contents to file #TODO - Add special lock file if($file = fopen($AdFilePath, 'wt')) { if(flock($file, LOCK_EX)) { fwrite($file, $string); flock($file, LOCK_UN); } else { echo("Unable to acquire lock to $AdFilePath"); } fclose($file); } else { echo("Unable to open $AdFilePath for writing"); } } else { #echo "Unable to connect to $url[host]
\n"; #File retrieval failed - exiting function pass back old values. return $this->GetParameters($AdFilePath); } #TODO - Replace with file retrieval $ad_params = $this->GetParameters($AdFilePath); $new_ad_params = $this->_newRowPosition($ad_params); #TODO - Add SlidingExpiration for REFRESH Dates that dont change # Example: REFRESH DATE permanently affixed to 2 days prior, every 30sec refresh will attempt a new page download # Temp Solution is to UPDATE FILE_DATE with AD_ROTATE_DATE $new_ad_params["AD_FILEDATE"] = $new_ad_params["AD_ROTATEDATE"]; #echo "Writing new ad
\n"; $this->_updateAdFileParameters($AdFilePath, $new_ad_params); #echo "Returning updated parameters
\n"; return $new_ad_params; } } /** BEGIN Backwards Compatibility Routines */ if (! function_exists('array_combine')) { function array_combine($keys, $values) { foreach($keys as $key) $out[$key] = array_shift($values); return $out; } } if (!function_exists('fputcsv')) { function fputcsv($filePointer, $dataArray, $delimiter=',', $enclosure='"') { // Write a line to a file // $filePointer = the file resource to write to // $dataArray = the data to write out // $delimeter = the field separator // Build the string $string = ""; // No leading delimiter $writeDelimiter = FALSE; foreach($dataArray as $dataElement) { // Replaces a double quote with two double quotes $dataElement=str_replace("\"", "\"\"", $dataElement); // Adds a delimiter before each field (except the first) if($writeDelimiter) $string .= $delimiter; // Encloses each field with $enclosure and adds it to the string $string .= $enclosure . $dataElement . $enclosure; // Delimiters are used every time except the first. $writeDelimiter = TRUE; } // end foreach($dataArray as $dataElement) // Append new line $string .= "\n"; // Write the string to the file return fwrite($filePointer,$string); } } if (!function_exists('fgetcsv')) { #TODO - Remove or ??? #Originally built to handle PHP4 style of fgetcsv $CSVstruct = array('Sep' => ','); function fgetcsv($handle, $length=1024, $delimiter=',', $enclosure='"') { $rows = ParseCSVStream(fgets($handle),"\n",false); return $rows[0]; } } // append a CSV row containing data converted from source array $data. Actual conversion is done by ExCSVRowWalker() function AppendCSVRow($data) { global $CSVstruct; array_walk($data,'ExCSVRowWalker'.($CSVstruct['OutEOL']=="\n"?'N':'')); $CSVstruct['File'] .= implode($CSVstruct['Sep'],$data).$CSVstruct['OutEOL']; } // returns an array of rows (each row being an array of columns) with parsed data. Returns false when conversion fails. Auto-detects Win/Mac/*nix newline characters, which are converted all to the sequence specified (defaults to $CSVstruct['OutEOL']) for your convenience. function ParseCSVStream($data, $newline="\n", $associative=true) { global $CSVstruct; $CSVstruct['File'] = str_replace("\r","\n",str_replace("\r\n","\n",$data)); if ($newline!="\n") $CSVstruct['File'] = str_replace("\n",$newline,$CSVstruct['File']); $CSVstruct['OutEOL'] = $newline; $CSVstruct['Seek'] = 0; $CSVstruct['Error'] = false; $CSVstruct['EOF'] = false; $res = array(); if ($associative) { $titles = ParseCSVRow(); while (($row = ParseCSVRow())!==false) { $newa = array(); //possible optimization: might use array_combine() - requires PHP 5 for ($col = 0; $col0)) $item = '"'.str_replace('"','""',str_replace($CSVstruct['OutEOL'],"\n",$item)).'"'; } function ExCSVRowWalkerN(&$item) { if (!is_numeric($item)&&(strlen($item)>0)) $item = '"'.str_replace('"','""',$item).'"'; } // read next line from buffer. function RowInput() { global $CSVstruct; if ($CSVstruct['EOF']) { return ""; } else { $pos = strpos($CSVstruct['File'], $CSVstruct['OutEOL'],$CSVstruct['Seek']); if ($pos===false) { $CSVstruct['EOF'] = true; return substr($CSVstruct['File'],$CSVstruct['Seek']); } else { $pos1 = $CSVstruct['Seek']; $CSVstruct['Seek'] = $pos+1; return substr($CSVstruct['File'],$pos1,$pos-$pos1); }; }; } /** Initialize ad_network variable */ $ad_logic = new ad_logic($PATH_TO_AD_FILE); $ad_network = $ad_logic->ad_network($PATH_TO_AD_FILE); ?>