At last a solution to the "Server redirected too many times" issue ?
Hi folks,
I know this topic has been discussed many times on this forum, but I don't find a solution. I am working on a specialized spider, which works fine with most of web sites. But some of them cause infinite server redirections.
After having worked touhgly on the problem, it appears to me that the causes can be :
1/ The server asks authentication
2/ The server tries to set cookies on the first page, and checks if the browser (or web client, whatever) accepts cookies in the redirected page, and redirects until the cookies are correctly set.
3/ The developer forgot a loop somewhere...
4/ Proxy settings
5/ Something else...
I eliminated the third case because I know the developer (me... ) and I check ed with him.
I eliminated the first case too, because I checked on many web sites that don't need authentication, so the problem doesn't spring from here (it could, but that case has already been solved in that forom).
I checked I have no proxyes...
It appears to me that it is a cookie problem, or something else.
So I wrote code that makes session management and keeps the cookies the server wants, but it still redirects. So, is it something else ?
I have a few sites like this, if somebody finds a solution to connect to http://195.46.221.246/DATA/FR/MAGASIN/PRODUIT/p_mag_ProdList.asp?ID=53064 for example, he will be very thankful... and genious !
Thanks and good luck !!!
Poe
