How can I access my Internet Explorer history?

Internet Explorer History Merge (Repost)

• First, before I start with the explanation, I just wanted to note that this is a rather complicated and technical question and likely won't be solved by 1 simple google search.. It may require intuition involving combinations of searches, etc... If you're not absolutely sure your method will work, please request clarification... Dont immediately post a guess as an answer. I'm in no real rush to get the answer, but would like it within 2 months from 11/27/02. Explanation: Operating systems involved: Windows 2000 Server Applications involved: Internet Explorer 6 My Goal: To take previously backed up Internet explorer history files which were originally located in "C:\Documents and settings\Username\Local Settings\History\History.IE5" and 'MERGE' them in with a current, up-and-running internet explorer 6 history... eg: once this project is complete, I want to be able to click the history button and see not only the history of the current windows installation, but the history that I had previously backed up.... The current history dates back 3 months, so I currently see "Today", "Yesterday", "3 weeks ago", etc up to 3 months ago. Assume the backed up history comes from before the year 2000, which would be more than the allowable 999 days of history in internet explorer, so I need to combine all previous backed-up history into 1 history "Folder" ... For example, "4 months ago"... What I have backed up and how I backed it up: The files I have backed up from the previous windows installation are a hirearchy of "index.dat"s underneath what appear to be timestamped dictories under the %USERPROFILE%\Local Settings\History\History.IE5\ path -- the timestamped directories look similar to "MSHist012002111420021115"... (%USERPROFILE% is an environment variable that expands to "C:\Documents and Settings\<Currently-Logged-On-Username>")... This is how the directory/file heirarchy appeared on the original Win2k Installation when I originally backed up the history files: %USERPROFILE%\Local Settings\History | |-desktop.ini |-History.IE5\ | |-desktop.ini |-index.dat |-MSHist012002111420021115\ | |-index.dat | |-MSHist012002111320021114\ | |-index.dat | |-(many more of these MSHIST directories with the only file inside being index.dat) I copied them from this path to another drive using the following command under a DOS box: (from %USERPROFILE%\Local Settings\History): xcopy *.* d:\savedir /s/e/v/h The directory and file structure under d:\savedir looked identical to %USERPROFILE%\Local Settings\History after the copy was complete. What I have tried (ie: What doesnt work :-)): I have attempted xcopying back all of the MSHist################## files and the index.dat's underneath from the backup d:\savedir back into the %USERPROFILE%\Local Settings\History\History.IE5\ directory. This does not work. It might be that the "Main" index.dat in the %USERPROFILE%\Local Settings\History\History.IE5\ directory is an index of all the timestamped directories themselves. This is right about where I get lost... Acceptable answers: 1) Methods of "reindexing" the MSHIST directories in the main index.dat file (if this is how IE history functions) and Methods of "combining" the index.dat's from inside the individual (older) MSHIST directories, in order to get the previous history to appear in 1 IE History "Folder". (if this type of combination of index.dat's is possible?). or 2) Any other method of somehow maniuplating the history folders to do what I've described above in the "My Goal" section... (I have several computers at my disposal, so the method can involve more than 1 Win2k system if necessary). or 3) URL(s) of tool(s) that will do specifically what I've described above in the "My Goal" section... (I've searched, and found none that do specifically what I'm looking for here...only tools that delete/clean history/cookies/etc) or 4) Some kind of proof that this simply can't be done. Thanks... Please request clarification with any questions... -nhdw

• Answer:

  Hi nhdw-ga, Glad to know that the solution works. :-D Here is a description of how it works : DISCLAIMER : The Internet Explorer History file format is undocumented and the following information is largely a result of my own research. I can make no guarantee as to its accuracy. It works for me though! :-) The IE History Index.dat files v5.2 =================================== The IE5 History file format has not been documented anywhere, but is is quite similar to the IE4 cache file format documented at : - MyFileFormats : Internet Explorer Temporary Internet Files Cache ( http://www.myfileformats.com/download_info.php?id=5670 ) Basically, the file is divided into three main sections : - The File Header : This section begins with the identifier string "Client UrlCache MMF Ver 5.2" and contains information such as the size of the file and the offset from which the HASH(see below) begins. - The HASH record : This is a 4KB record which preceeds the URL records in the file.This hash is used by Internet Explorer to speed up its traversal/sorting of all the URL records contained in the file. A file can contain more than one HASH records. This hash is generated by IE using an undocumented algorithm, and hence cannot be easily(if at all) duplicated. - The URL record : Each HASH record is followed by one or more URL records. Each URL record has a size which is a multiple of 128 bytes. The actual URL starts in the URL record at the 97th byte, and is stored as a null terminated string. Each URL is preceeded by a prefix which is usually 19 bytes long. Creating an index.dat ===================== There are two main problems we face when trying to create an index.dat file by ourselves. Firstly, large parts of the file header are undocumented and we have no idea what information they store. Secondly, for every URL we add to the file, we must modify the HASH record accordingly so that IE can access the URLs. But we cannot do this, as we do not know the algorithm being used to generate the HASH. If IE detects an inconsistent index.dat, it deletes the file and replaces it with an empty index.dat containing only the file header. Since my attempts to create index.dat files ended in failure, I decided to attack the problem in another way. Internet Explorer - Registry settings ===================================== A list of registry accesses made by Internet Explorer showed an interesting thing; IE was reading the path to the History files from the registry! This lead me to the following registry key, where IE stores its list of History files : HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Internet Settings\5.0\Cache\Extensible Cache\ This registry key contains a subkey, which ususally has a name in the format : MSHist01yyyymmddYYYYMMDD where yyyymmdd : The date from which the URLs are indexed (From:) YYYYMMDD : The date to which the URLs are indexed (To:) You can see that this format is the same as the subfolder names in your \History\History.IE5 folder. One such subkey is present for every subfolder. Each of the subkey contains the following data : - CacheLimit : usually 8192 (file limit in KB or no. of URLs ?) - CacheOptions : usually 11 (unknown) - CachePath : path to the folder which contains the index.dat file for this duration. This path can point to any folder on disk which contains a valid History index.dat file. - CachePrefix : This stores the prefix string which preceeds the actual URLs in the URL records(see description above). So, in order to add a particular index.dat to the current IE History, we have to create the relevant subkey for it and provide the time interval for which it contains the History, and the path to the file. If the 'To' date for the time interval falls out of the time limit specified in Internet Explorer's Options, IE deletes the specified file and the registry entry. The two History Index.dat files =============================== You will notice that your History.IE5 folder contains an index.dat file. This file is different from those contained in the subfolders of History.IE5. This main index.dat is not specified in the registry, and is unique, i.e. there can only be one such index.dat. This also means that the data contained in the main index.dat that you backed up, cannot be imported by adding it to the registry. If you try to do that, IE detects it as a corrupt file and deletes it. The only way (that I was able to determine) to get the data out of this file is to extract the URLs from it and save them as URL shortcuts. Creating URL shortcuts ======================= The url shortcuts that appear in the Favourites folder, are simple text files containing data in a fixed format. In order to create a URL shortcut, follow these steps : - Create a new text file - Enter the following text in the file : [InternetShortcut] URL=http://www.someaddress.com/ - Now save the file, and change its extension from .txt to .url Windows automatically treats files with a .url extension as Internet shortcuts and extracts the URL from it. You can find detailed info on the URL file format at : - MyFileFormats : Internet Explorer URL files ( http://www.myfileformats.com/download_info.php?id=7140 ) To create all the URL files, my program parses the index.dat files, looking for URL records. From them, it extracts the URL string, and strips away the cache prefix string and other prefixes from it. It then writes this data into URL file in the specified format. Naming the URL file creates a slight problem, since the URL contain many characters (such as / ? :), which are not allowed in a Windows filename. I get around this by replacing all the invalid characters with a dash(-) character. =================================== Hope this helps. :-) If you need any clarificatons, just ask! I will be putting up a new version of my app for download in a few days, so do check it out. Regards Theta-ga