Request website title - Save title in local file

andrew.smith · December 1, 2020, 7:37pm

I am looking for some help to write a simple PHP function which will make request to website and get title from title tag and save the title in local file, if I make a request again for same link, I will get title from local file and display it.

I found this for a starting point. but I am very new to php. can someone help please. Thank you so much.

function website_title($url) {
   $ch = curl_init();
   curl_setopt($ch, CURLOPT_URL, $url);
   curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
   // some websites like Facebook need a user agent to be set.
   curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.94 Safari/537.36');
   $html = curl_exec($ch);
   curl_close($ch);
   $dom  = new DOMDocument;
   @$dom->loadHTML($html);
   $title = $dom->getElementsByTagName('title')->item('0')->nodeValue;    	
	$noaccess= array('404', '403');
$url_string = end(explode(' ',$title));
if (in_array($url_string,$noaccess)){
$title = $url;
} else 
	
{    
}
 return $title;    	
	}

whispers · December 1, 2020, 10:46pm

I dont understand what you want to do when you say this:

" if I make a request again for same link, I will get title from local file and display it."

You just want to display the title from the .txt file? (is that it?)

Since you are using the URL as your starting point…

You will probably need to save that as well… other wise if you only check the ‘file’ for the title… it will be too late, as you would have already made the call using the URL to get the ‘title’… to check against what is saved in the ‘text file’…

IMO you would need:

input field where you enter in your ‘URL’ to get title of.
when you hit submit… it checks your local .txt file (or database…or whatever) to see if that same URL is in there… if so… it returns the associated title that matched that URL in the text file.
if no match is found… then it continues to create the DOM element, scrapes the ‘title’ tag and returns a title for you…
you then take this returned value/string and write to a text file on a new line

andrew.smith · December 2, 2020, 2:27pm

Yes, That I would like that to hapend but I am new to php and I am not sure how would go about it.

whispers · December 2, 2020, 2:47pm

What have you tried? Where are you at currently?

We all LOVE to help/teach around here… but its something we do for free/fun as a benefit. (Not so much about just doing it for someone, so they didnt do or learn anything).

Show us what you got…

andrew.smith · December 10, 2020, 4:15pm

I paid someone to write for me. But here it is. Might be usefull for people to have.

function page_title( $url ) {

    $read = file_get_contents(dirname(__FILE__) . "/database.txt");
    $lines = explode("\n", $read);

    if ($lines) {
        foreach ($lines as $data) {
            if (trim($data) == "") {
                continue;
            }

            $d = explode("|", trim($data));
            if ($d[0] == $url) {
                return $d[1];
                exit();
            }
        }
    }        

    $headers = ['Content-Type: text/html'];

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Linux; Android 10; SM-A205U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Mobile Safari/537.36.");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    $output = curl_exec($ch);
    curl_close($ch); 

    $res = preg_match("/<title>(.*)<\/title>/siU", $output, $title_matches);
    if (!$res) 
        return null; 

    $title = preg_replace('/\s+/', ' ', $title_matches[1]);
    $title = trim($title);

    if ($title) {
        $write = fopen(dirname(__FILE__) . '/database.txt', 'a');
        fwrite($write, $url . '|' . $title . "\n");  
        fclose($write);  
        return $title;         
    }

}

andrew.smith · December 10, 2020, 4:28pm

function page_title( $url ) {	
		
	$cache_directory = '/tmp/cache';
	$cache_file = $cache_directory . '/urldatabase.dat';
if (!file_exists($cache_file)) {
    touch($cache_file, strtotime('-1 days'));
}
        $read = $cache_file;	
        $lines = explode("\n", $read);
        if ($lines) {
            foreach ($lines as $data) {
                if (trim($data) == "") {
                    continue;
                }

                $d = explode("^", trim($data));
                if ($d[0] == $url) {
                    return $d[1];
                    exit();
                }
            }
        }        

        $headers = ['Content-Type: text/html'];

        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
        curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Linux; Android 10; SM-A205U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Mobile Safari/537.36.");
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        $output = curl_exec($ch);
        curl_close($ch); 

        $res = preg_match("/<title>(.*)<\/title>/siU", $output, $title_matches);
        if (!$res) 
            return null; 

        $title = preg_replace('/\s+/', ' ', $title_matches[1]);
        $title = trim($title); 

        if ($title) {
            $write = fopen(dirname(__FILE__) . $cache_file, 'a');
            fwrite($write, $url . '^' . $title . "\n");  
            fclose($write);  
            return $title;         
        }

    }