• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

Item4267: Network problem reaching bugs web when saving or attaching.

Item Form Data

AppliesTo: Component: Priority: CurrentState: WaitingFor: TargetRelease ReleasedIn
NotTWiki Bugs Normal No Action Required TWiki:Main.PeterThoeny n/a  

Edit Form Data

Summary:
Reported By:
Codebase:
Applies To:
Component:
Priority:
Current State:
Waiting For:
Target Release:
Released In:
 

Detail

it happens to me again and again and again

I edit a bug item. And when I save it the browser never gets a reply from the server and the item is not saved.

I can try again and again and again and I can never save the item no matter what I try. I can close the browser and try again. I can use Firefox instead of IE. It does not matter. Once it goes in orbit it never wants to save that item.

But I can still view and edit. it is the save script that never returns.

I only see it here on bugs but it could be a severe problem. I simply never test much on other TWikis running MAIN here from work.

I have seen it from home a few times but I mainly see it here from Motorola.

I have often successfully opened the same item in an anonymizer and then I could save.

Is there an issue with sessions and IP addresses maybe?

-- TWiki:Main/KennethLavrsen - 18 Jun 2007

Editing again just to check. If you read this then it went well.

The item that just failed is Item4173

-- TWiki:Main/KennethLavrsen - 18 Jun 2007

Tried with closed browser and reopen and I still cannot save Item4173.

-- TWiki:Main/KennethLavrsen - 18 Jun 2007

Managed to edit Item4173 using an anonymizer again so it it editing from a different IP address. Will now try directly again.

-- TWiki:Main.KennethLavrsen - 18 Jun 2007

Could still not edit Item4173 from here. Very strange. Will have to study the server log later (cannot login with ssh from here). Time stamp is 19:30 Danish time.

-- TWiki:Main.KennethLavrsen - 18 Jun 2007

Now home trying to analyse.

I have found something interesting. Look at the log from develop looking at all those entries where I was authenticated.

129.188.69.169 - KennethLavrsen [18/Jun/2007:12:06:26 -0500] "GET /~twiki4/cgi-bin/edit/Bugs/Item4173?t=1182186184 HTTP/1.1" 200 38868 "http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4173" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
136.182.158.137 - KennethLavrsen [18/Jun/2007:12:10:03 -0500] "GET /~twiki4/cgi-bin/edit/Bugs/ItemAUTOINC1?templatetopic=WebTopicEditTemplate;formtemplate=ItemTemplate;_T=120929 HTTP/1.1" 200 30135 "http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/WebHome" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
136.182.158.137 - KennethLavrsen [18/Jun/2007:12:13:49 -0500] "POST /~twiki4/cgi-bin/save/Bugs/ItemAUTOINC1 HTTP/1.1" 302 - "http://develop.twiki.org/~twiki4/cgi-bin/edit/Bugs/ItemAUTOINC1?templatetopic=WebTopicEditTemplate;formtemplate=ItemTemplate;_T=120929" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
136.182.158.137 - KennethLavrsen [18/Jun/2007:12:13:58 -0500] "GET /~twiki4/cgi-bin/edit/Bugs/Item4267?t=1182186831 HTTP/1.1" 200 31021 "http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4267" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
136.182.158.137 - KennethLavrsen [18/Jun/2007:12:14:51 -0500] "POST /~twiki4/cgi-bin/save/Bugs/Item4267 HTTP/1.1" 302 - "http://develop.twiki.org/~twiki4/cgi-bin/edit/Bugs/Item4267?t=1182186831" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
136.182.158.137 - KennethLavrsen [18/Jun/2007:12:16:10 -0500] "GET /~twiki4/cgi-bin/edit/Bugs/Item4173?t=1182186957 HTTP/1.1" 200 38868 "http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4173" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
217.147.104.223 - KennethLavrsen [18/Jun/2007:12:17:54 -0500] "GET /~twiki4/cgi-bin/edit/Bugs/Item4267?t=1182187050 HTTP/1.1" 200 31218 "http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4267" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
217.147.104.223 - KennethLavrsen [18/Jun/2007:12:18:30 -0500] "POST /~twiki4/cgi-bin/save/Bugs/Item4267 HTTP/1.1" 302 - "http://develop.twiki.org/~twiki4/cgi-bin/edit/Bugs/Item4267?t=1182187050" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
62.243.208.121 - KennethLavrsen [18/Jun/2007:12:22:36 -0500] "GET /~twiki4/cgi-bin/edit/Bugs/Item4173?t=1182187319 HTTP/1.1" 200 38868 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
62.243.208.121 - KennethLavrsen [18/Jun/2007:12:24:34 -0500] "POST /~twiki4/cgi-bin/save/Bugs/Item4173 HTTP/1.1" 302 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
62.243.208.121 - KennethLavrsen [18/Jun/2007:12:27:10 -0500] "POST /~twiki4/cgi-bin/save/Bugs/Item4267 HTTP/1.1" 302 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
129.188.69.169 - KennethLavrsen [18/Jun/2007:12:28:18 -0500] "GET /~twiki4/cgi-bin/edit/Bugs/Item4173?t=1182187680 HTTP/1.1" 200 39150 "http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4173" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
129.188.69.169 - KennethLavrsen [18/Jun/2007:12:30:14 -0500] "POST /~twiki4/cgi-bin/save/Bugs/Item4267 HTTP/1.1" 302 - "http://develop.twiki.org/~twiki4/cgi-bin/view/Bugs/Item4267" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)"
[root@develop httpd]#

Notice: First time I edit Item4173 I have IP address 129.188.69.169.

2nd time I try to do something is when I open a new bug item. And I do that because I could not save. There is no authenticated POST at all.

Where does that disappear?

When I try again our firewall has chosen a different gate: 136.182.158.137

Then I again try and again with a new IP. 217.147.104.223

And fail.

Then I try with the anonymizer: 62.243.208.121 (that is actually my home server where I find it useful to have this tool - it is not open for public access).

And I succeed like I have so many times before when I run into the problem.

And then finally try again directly from our firewall gateway: 129.188.69.169 and fail again.

I am not much closer to an explanation. Need to look at detailed logs more

-- KennethLavrsen - 18 Jun 2007

Observed this again. And this time from home.

I could not edit until I used a different browser on a different ADSL line (good I have two).

I was viewing the access_log on develop.twiki.org and it never logged the save. I tried many times. So it is not a twiki code issue but an issue with apache.

Is there some crazy setup I have overlooked that tries to keep certain traffic out?

I never observed this on my own servers running MAIN.

-- TWiki:Main.KennethLavrsen - 27 Jun 2007

I've also not observed this when trying to save here on the Bugs system from my computers at home frown

Sounds like a really painful issue.

One long shot thought - you have mentioned previously that you have more than one line going out. is it at all possible that your web traffic is alternating between them, and (ok, more IF's) that you are using NAT, so your POST and your authentications are actually from 2 IP's - thus are different sessions on TWiki - as TWiki sessions do have some 'ensure its the same source IP' magic...

yep - that's an extreme long shot frown

-- TWiki:Main.SvenDowideit - 27 Jun 2007

I have had the same problem the few times I have been editing during the last weeks. I have only the same line, so I just try repeatedly (up to 7-9 times) and in the end I somehow manage to save.

Some kind of a "buffer" mechanism for hits - meant to queue them up but instead throws them away - could be a cause? I have no knowledge of any load balancer or similar devices involved here, though.

-- TWiki:Main.SteffenPoulsen - 28 Jun 2007

ok, so we have a second (yay) - when either of you say 'save' are you using comment? or editing (or both).

-- TWiki:Main.SvenDowideit - 28 Jun 2007

I cannot remember if I had the problem with Comment plugin. I may have.

But for sure it happens with normal edit/save.

When I connect from home I am 100% sure that all traffic goes through only one router. It is defined by the default gateway in the windows network setting.

From Motorola it could be the case that IP address changes. But I doubt this is the issue. There is something else and I cannot figure it out.

No matter how locked the saving is I can always view and even edit any other topic. Or same topic that I work on. It is the save script that fails and it is not the script itself that fails. It never gets called by Apache. It seems Apache sometimes decides to block saving.

I am happy to see that Steffen has seen it too. I feel less alone now wink

As I save this from work I have the problem again. And again I am saving through an anonymizer proxy. This is really annoying

-- TWiki:Main.KennethLavrsen - 28 Jun 2007

Trying again from work

-- TWiki:Main.KennethLavrsen - 28 Jun 2007

Strange. Once the topic has been saved from another IP I can save again from work. This time I tried both comment and normal edit. Both worked.

-- TWiki:Main.KennethLavrsen - 28 Jun 2007

ooo, good observation - so it might be the leases that are causing you issues..

next time you decide to have this problem >groan< take a peek at the lease file, (or even better, look for the lease before you hit edit) - it might even be a left over from something.

i'll bet if this gets figured out, i'd even let you buy me a coffee in Rome :}

-- TWiki:Main.SvenDowideit - 28 Jun 2007

Most of the incidents have been when I am at work and I cannot ssh out through our firewall. So I have to try more from home.

-- TWiki:Main.KennethLavrsen - 29 Jun 2007

Editing this only to provoke the error and learn from it

-- TWiki:Main.KennethLavrsen - 30 Jun 2007

This continues to happen to me daily and I am fed up with it. Each time it happens I have to give up saving a bug for hours or I have to setup a machine to use a different IP address and then I can save the same bug without delay

WHO HAS THE CONTACT TO FASTSERVERS?

It seems noone in charge of develop.twiki.org is doing anything about this. This is not the d.t.o server that is setup wrong. There is some server between the internet and d.t.o that queue up POSTs in some cases. I can see it on the Apache log. The entry arrives minutes later. It is not buffering delay in Apache. It shows all other connections instantly. I can always GET. It is always POST that gets delayed. Ie. either when you save a bug or when you attach a file. And often when the topic has reached a certain length. It almost never happens with a small new topic. So it may have something to do with the package stream being broken up and the last parts arriving late.

This happens to me both at Motorola where my IP address is a proxy in Germany and from home and other have seen it too here. So it is not my ISP that has a problem. It is Fastservers that have a network problem.

And they will never fix it is noone reports it to them.

So who is doing that?

Elevating to urgent because this shit problem cuts by productivity to half some days. This morning I spent 20 minutes trying to save Item4570. Last night it was another bug that tool me 15 minutes. And yesterday from work when I had 15 minutes to work on TWiki I had to give up.

Please Peter or Sven or whoever has the contact to Fastserver. Report this problem.

-- TWiki:Main.KennethLavrsen - 16 Oct 2007

To give a more geographical informtaion on this, I have seen this problem in the past but very, very rarely (once in every couple of hundred saves, I would guess, and not for a long time). I connect to the internet via my ISP in Sheffield (England).

-- TWiki:Main.CrawfordCurrie - 17 Oct 2007

If possible this should be verified with tcpdump (preferable in both ends of the tcp packet stream) before it is reported - such information is valuable / required for the people who are going to have to look into this in the end. Apache logs "late" - this needs to be debugged also with something that logs "early".

-- TWiki:Main.SteffenPoulsen - 18 Oct 2007

I haven't seen this since I last commented (I've been watching out for it) so I have to conclude it's related to the client end. Changed the headline and attribution accordingly. I would be happy to assist anyone trying to debug at the d.t.o end.

CC

Client end??

Steffen sees the problem. I see the problem from 3 different computers on two different ADSL lines at home. And I have the problem at work in a 4th computer. And I see the problem in both IE and FF. Strange that so many clients have the same problem.

I have been monitoring the apache log live while having the problem. The GETs get through instantly. The POSTs are sometimes delayed. Delayed by minutes. And it happends only at d.t.o. Not at t.o (it has other problems not responding to anything once in a while).

The thing with the POSTs is that the messages sent to the server are significantly longer. My feeling is that when you do a GET you send very few bytes of information. When you POST you submit a topic or you send a file. I bet the packages get split up somewhere in the routing from the client to the server. And this both from Denmark and Germany (our proxy is in Germany). I bet fastserver has a problem with how things are routed to them or they have a load sharing thing that goofs up and split the packages wrongly. And it may be depending on which IP range the data comes from since Crawford does not see it. It seems Peter does not see it either.

It comes in waves. There are days where it does not happen. And there are days where working on d.t.o is impossible. This is why we need Fastservers to look at this.

I changed the headline to make Peter (who has the contact to Fastservers) but I did not see any reaction. Now this issue is toned down again so it can be ignored like it has been since June.

Debugging requires that you have the problem and before I have ever been able to setup a big debugging session the problem is gone.

I guess I have to contact Fastservers then since those that do not have the problems seems to care very little to get it solved. Can I have the contact details please?

-- TWiki:Main.KennethLavrsen - 04 Nov 2007

Does not seem to be an issue anymore. Closing stale ticket...

-- TWiki:Main.PeterThoeny - 2010-11-06

ItemTemplate
Summary Network problem reaching bugs web when saving or attaching.
ReportedBy TWiki:Main.KennethLavrsen
Codebase ~twiki4
SVN Range TWiki-4.1.2, Thu, 14 Jun 2007, build 14128
AppliesTo NotTWiki
Component Bugs
Priority Normal
CurrentState No Action Required
WaitingFor TWiki:Main.PeterThoeny
Checkins

TargetRelease n/a
ReleasedIn

Edit | Attach | Watch | Print version | History: r15 < r14 < r13 < r12 < r11 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r15 - 2010-11-06 - PeterThoeny
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback