Find solution
How can I make a dynamic web page in PHP?

Is it possible to retrieve source code of a redirected web page that uses sessions and cookies with PHP?

  • There is a web page that I want to scrape some elements from but the  page requires the viewer to click a link to get to the page. The url is like this: http://exampledomain.com/gp/bestsellers/books/10578201/375-9114798-0670610 If I access that page, it displays a link to confirm the visitors age to check if the visitor is matured enough to view the contents. The link url is like this: http://exampledomain.com/gp/product/black-curtain-redirect.html?redirectUrl=%2Fgp%2Fbestsellers%2Fbooks%2F10578201 If the link is clicked, it redirects back to the original page and the actual contents are shown, http://exampledomain.com/gp/bestsellers/books/10578201/378-5247603-7740217?&redirect=true I'm guessing that the server uses sessions and cookies or referrer to decide whether it redirects to the destination page. So is it possible to retrieve the source code of such a page?

  • Answer:

    I'm not 100% sure that I understand the question, but I'll try to answer... In short, yes. You will probably need to use cURL or something similar though. Please see the bottom answer here: http://stackoverflow.com/questions/3519939/make-curl-follow-redirects

Marty Gottesfeld at Quora Visit the source

Was this solution helpful to you?

Other answers

Yes it is possible, you just need to track the cookies that the server sets in the age verification form, and add them to the request headers of the following pages. google chrome will help you write your curl command, as you can copy any request sent from your browser as a curl command with all its headers attached already!see: http://stackoverflow.com/questions/15995919/curl-how-to-send-cookies-via-command-linehttp://www.lornajane.net/posts/2013/chrome-feature-copy-as-curl

Ahmad Moqanasa

You're going to have to use a browser.The way to do this is to use a library like Selenium or PhantomJS to open the first URL, click the redirect on the page and then get data programmatically.

Hugo Di Francesco

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.