Viewing Untouched Web Page Source Code

When you use your browser to view the source code of a page, what you see may be different than what was sent to the browser in the first place.

Which may be what you want. The developer tools that come with recent browsers make it relatively easy to see the exact code as it exists at specific sections of a web page.

But if you want to see what the server sent to the browser in the first place — to see what search engines see, for example, or to study how the page is coded — there is no confidence that the source code the browser presents now is what it originally received.

That's because when the server has sent the web page source code to the browser, JavaScript can change the source code.

Changes by JavaScript either may be delayed or may occur as soon as the webpage arrives at the browser (or even before the page has finished arriving). Further, Ajax can bring in source code that replaces parts of the current page, also immediately or delayed — or both. Even the entire web page can be switched.

If the web page specifies no-cache, some or all of the source code might not be available at all from the browser.

This article provides a tool that allows you to see web page source code exactly as a browser would receive it from the server.

The tool is simple to use.

Type in a URL and tap the button. The tool will respond with the source code of the web page at the URL — the source code as it was received from the server. (More information below this live interface.)

The user-agent string is pre-filled with the identification of the browser you are using. Optionally, you can change that to see how the server responds to a different browser. (Some servers respond differently to mobile device browsers than they do to desktop computer browsers.) A resource for lists of user-agent strings is this website.

Also optionally, you can check the checkbox to see what header information the server sends to the browser.

Source Code

You are welcome to use the form on this page. Or, you may implement the system on your own website.

Implementation is in two parts: (1) The form (like you used above) and (2) the PHP software that grabs the web page.

Let's install the PHP software first.

Here is the PHP software source code. No customization is required.

<?php
/*
Get Web Page Source Code From Server
Version 1.0
July 10, 2021
Will Bontrager Software LLC
https://www.willmaster.com/
*/
if( empty($_POST['url']) ) { echo 'Inappropriate access.'; exit; }
if( empty($_POST['ua']) ) { $_POST['ua'] = 'tester'; }
$header = $page = '';
$info = GetWebPage($_POST['url'],$header,$page);
echo "<p style='margin:0 0 0 1em; text-indent:-1em;'>Response from server for:<br>{$_POST['url']}</p>\n";
if( $info['http_code'] != 200 ) { echo "<p>Response code: {$info['http_code']}</p>\n"; }
if( $info['errno'] > 0 ) { echo "<p>Error: {$info['errno']} => {$info['errmsg']}</p>\n"; }
else
{
   if( isset($_POST['header']) and $_POST['header'] )
   {
      echo '<p style="margin-bottom:0;"><b>Header returned by server:</b></p><div style="white-space:pre-wrap;">';
      echo htmlspecialchars($header);
      echo '</div>';
   }
   echo '<p style="margin-bottom:0;"><b>Page source code:</b></p><div style="white-space:pre-wrap;">';
   echo htmlspecialchars($page);
   echo '</div>';
}
exit;

function GetWebPage($url,&$header,&$page)
{
   $options = array(
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_HEADER         => true,
      CURLOPT_CONNECTTIMEOUT => 120,
      CURLOPT_TIMEOUT        => 120,
      CURLOPT_USERAGENT      => $_POST['ua'],
      CURLOPT_VERBOSE        => false
   );
   $info = array();
   $ch = curl_init($url);
   curl_setopt_array($ch,$options);
   list($header,$page) = preg_split('/\r?\n\r?\n/',curl_exec($ch),2);
   if( strpos(trim($page),'HTTP/')===0 ) { list($header,$page) = preg_split('/\r?\n\r?\n/',$page,2); }
   $err = curl_errno($ch);
   $errmsg = curl_error($ch) ;
   $info = curl_getinfo($ch);
   curl_close($ch);
   $info['errno'] = $err;
   $info['errmsg'] = $errmsg;
   return $info;
}
?>

Save the PHP software as SourceCodeGetter.php (or other *.php file name, if you prefer) and upload it to your server. Make a note of its URL.

Now, here is the source code for the form you used further above. Notes follow.

<div style="border:3px solid #ccc; border-radius:.5em; padding:.5em; max-width:500px;">
<!-- Because this is Ajax only, no HTML <form...> tag needs to be used. -->
<!-- To prefill the user-agent string, this form must be on a PHP web page. -->
<p style="margin-top:0;">
The URL:<br><input type="text" id="source-code-url" style="width:100%;">
</p>
<p>
The user-agent browser identification:<br><input type="text" id="source-code-user-agent" value="<?php echo($_SERVER['HTTP_USER_AGENT']) ?>" style="width:100%;">
</p>
<p>
<input type="checkbox" id="source-code-headers">&thinsp;Include headers.
</p>
<p>
<input type="button" value="Get" style="width:100%;" onclick="SourceCodeGetAtURL()">
</p>
<div id="source-code-source-code" style="border-top:1px solid #ccc; padding-top:.5em; overflow:auto;">
(Source code will appear here.)
</div>
<script type="text/javascript">
function SourceCodeGetAtURL()
{
   var sourceCodeGetterURL = "/php/SourceCodeGetter.php";
   http = new XMLHttpRequest();
   if(! http) { return false; }
   var params = new Array();
   params.push( "url=" + encodeURIComponent(document.getElementById("source-code-url").value) );
   params.push( "ua=" + encodeURIComponent(document.getElementById("source-code-user-agent").value) );
   params.push( "header=" + (document.getElementById("source-code-headers").checked?'1':'0') );
   http.onreadystatechange = function()
   {
      if(http.readyState == 4 && http.status == 200)
      {
         document.getElementById("source-code-source-code").innerHTML = http.responseText;
      }
   }
   http.open("POST",sourceCodeGetterURL,true);
   http.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
   http.send(params.join("&"));
}
</script>
</div>

Notes —

About halfway down the above source code, you'll see the JavaScript that is the Ajax engine.

In the JavaScript, replace /php/SourceCodeGetter.php with the URL of the PHP software that grabs the web page, SourceCodeGetter.php, that is installed on your server.

Because Ajax is protocol, domain name, and port number sensitive, specify the URL as a relative URL. That way, all those sensitive parts will always match.

A relative URL, in this case, is simply the absolute URL with the protocol and domain name parts removed. As an example, absolute URL https://example.com/page.php becomes /page.php as a relative URL.

When /php/SourceCodeGetter.php has been replaced with the correct URL, paste the above form source code into a web page.

Put the web page into your browser and you are good to go.

You are now able to view source code as it is sent by the server, rather than as the browser might end up having changed it.

(This article first appeared with an issue of the Possibilities newsletter.)

Will Bontrager