UtterAccess HomeUtterAccess Wiki

Welcome Guest ( Log In | Register )

Custom Search
Edit Discussion
> WebpageToString    
WARNING! For many web sites, the practice of screen scraping data is illegal. Ensure that the method described below is used with care and in a fully legal manner.

This function uses the MSXML HTTP Request class and scrapes the html content from the specified web page/query and returns the results in a VBA string.

Example Usage:
MyString = WebpageToString("http://SomeWebsite.com/SomePageIWant.php?ID=165484")

CODE

' WebpageToString
' http://www.utteraccess.com/wiki/index.php/WebpageToString
' Code courtesy of UtterAccess Wiki
' Licensed under Creative Commons License
' http://creativecommons.org/licenses/by-sa/3.0/
'
' You are free to use this code in any application,
' provided this notice is left unchanged.
'
' rev  date                          brief descripton
' 1.0  2012-01-17                    
'
Public Function WebpageToString(ByVal srcURL As String) As String
 'return a string of the specified URL source
 'prelim version, no validations
 On Error Resume Next
 Dim xhr As Object
 Set xhr = CreateObject("Microsoft.XMLHTTP")
   xhr.Open "GET", srcURL, False
   xhr.Send
   WebpageToString = xhr.responsetext
 Set xhr = Nothing
 If Err.Number <> 0 Then
   WebpageToString = ""
 End If
End Function

Forcing a Page Refresh

In repeated attempts to get updated information from a server, the XMLHTTP class may return cached copies of the page. To "trick" the library into thinking it needs a new page each time, we can append a unique random string to the end of the URL each time the request is made (use the format of URL & "?asdfghjkl"). Formatting the current data/time at the time of calling will generally suffice.

Example:

CODE
?WebpageToString(SomeURL & "?" & Format(Now(), "yyyymmddhhmmss"))


Creative Commons License
WebpageToString by UtterAccess Wiki is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.
Editing and revision of the content is freely encouraged; for details, see Expected Usage.

Edit Discussion
Custom Search
Thank you for your support!
This page has been accessed 4,716 times.  This page was last modified 12:39, 11 January 2013 by Jack Leach.   Disclaimers