Item4570: The check on DefaultUrlHost should allow 'www' in front of a url

http://asapframework.org

http://www.asapframework.org

But both urls are valid. I don't have any control over which url is used by users.

My LocalSite.cfg says <code>$TWiki::cfg{DefaultUrlHost} = 'http://asapframework.org';</code>

I suggest to loosen the check on the url to ignore any 'www' in the url.

-- TWiki:Main/ArthurClemens - 07 Sep 2007

Seems to work OK at the moment? A template login is spawned "extra" when coming in from www.asapframework.org but that seems to be it (configuration issue?). Cancelling the the edit from the www side does give a:

Access check on Main.WebHome failed.
Action "redirect": unsafe redirect to http://www.asapframework.org/wiki/bin/view/ASAP/WebHome: host does not match {DefaultUrlHost} "http://asapframework.org". 

I don't think twiki.org / www.twiki.org has this problem, and in general a TWiki should just service any any url already. Perhaps this change is part of latest cross side scripting patches?

I notice my <nop>UserForm doesn't have any information upon registration either - looks suspicious.

-- TWiki:Main.SteffenPoulsen - 07 Sep 2007

Or is the error coming from configure?

-- TWiki:Main.CrawfordCurrie - 08 Sep 2007

twiki.org / www.twiki.org may not have this problem yet. asapframework.org runs on the latest svn version. I still don't have a solution for this.

-- TWiki:Main.ArthurClemens - 08 Sep 2007

Is this really a release blocker?

I think not.

Lowering to normal. There is an Apache work around. Rewrite the www to without www.

-- TWiki:Main.KennethLavrsen - 08 Sep 2007

we can't just blindly allow www.somedomain.org and somedomain.org, as this might actually be the result of a dns attack - they don't magically point to the same server, as they are seperate entries in the dns setup.

Also, to make life even more painful - what about IP addresses? I'm constantly getting bugged by setting the twiki conf to http://localhost, and then later if i access it via IP (either the dhcp allocated one, or 127.0.0.1) this problem raises its head.

so... the fix would be,

• leave $TWiki::cfg{DefaultUrlHost} as this is the default
• add a list of valid addresses that are acceptable, including {ipof:server.something.org} - maybe.

-- TWiki:Main.SvenDowideit - 15 Sep 2007

Hmm, I can't seem to find the Codev topic / Bugs item in which this were discussed / changed?

TWiki always just answered any urls that were coming in.

I don't think it is appropriate to filter on this in TWiki - it is an option that belongs in the webserver setup (i.e. apache vhost config). But I assume previous arguments on this item make sense, I hope someone has a pointer.

-- TWiki:Main.SteffenPoulsen - 15 Sep 2007

Apparently Item3580 was the motivation behind this one. Tries to solve an "outgoing" redirect.

A possible alternative solution might be to check that the incoming and outgoing URI hosts match in isRedirectSafe (ignoring the DefaultUrlHost part)? (A whitelist doesn't sound that bad either).

The error text could possibly be updated to mention the AllowRedirectUrl parameter for intranet sites.

-- TWiki:Main.SteffenPoulsen - 15 Sep 2007

mmm, I'm trying to think of a way that someone can fake the wrong host, so that the redirect goes to the wrong server (by using the in url)

and I think you're right, we can 'allow' those without adding the whitelist.

I'll implement that and see

ooo, would you look at the comment I left last time i was there...

    #TODO: this should really use URI
    #TODO: this should also grok aliases for the current host.
    #(127.0.0.1, ip, multi-homed, localhost etc) though this raises the danger level somewhat. following the link to [[Item3580]] , no, thats not the reason for this issue, that bug was just brining together the redundant implementations into one - the root cause of this, was a hack attack (phishing) report.

-- TWiki:Main.SvenDowideit - 15 Sep 2007

mmm, actually, if we're getting a middleman attack, then we should not magically accept the domain that is being sent, as that will just propagate the problem.

SO, I'll do just the whitelist, as an expert setting

-- TWiki:Main.SvenDowideit - 17 Sep 2007

Please elaborate - I don't understand the concept of a middleman attack here. Any domain validation belongs in the virtual host mechanism only imho.

-- TWiki:Main.SteffenPoulsen - 17 Sep 2007

several quickies come to mind

1. if you accept any SERVER_ADDR that got to your server, you're allowing someone else to take control of the known DNS for your twiki, which they can later use to redirect, or intercept your traffic
2. not middleman, but it would allow hackers to phish the TWiki (rather than the other way around), again, because you've allowed them to use your server from theur domain
3. then of course, the true middleman, wouldn't need our help, cos they can just re-write the html on the fly to always go through their site

salient point is, we can never accept arbitary input from users without checking it - and in this case, that means checking the redirect url. Don't forget that SERVER_ADDR comes from the user, so is spoofable too

commited.

Arthur, you do have control over the URL - most sites redirect one url to the other, so that things are consistent - but now you should be able to add the 'other' hostname to the expert setting {PermittedRedirectHostUrls} (expert, security)

-- TWiki:Main.SvenDowideit - 18 Sep 2007

I do not accept "any SERVER_ADDR that got to your server" - I just want to filter this where it is relevant, which is in the webserver config. Having to set this in several places just doesn't make any sense.

It is in vhosts we already have the flexibility for all kinds of tricks for regex-matching the hostname - if we must have a local mechanism for limiting answers it must be "*" per default.

-- TWiki:Main.SteffenPoulsen - 18 Sep 2007

you don't accept well known spoofing attacks (like faking SERVER_ADDR)? and dns attacks?

Either you're not making sense to me, or you've got alot more reading to do on what hackers get upto.

I'm somwhat confused though, it seems to me that you've lost sight of the original bug that Arthur reported - he's not using vhosts, or anything unusal, nor complex.

the redirect code was locked down from '*' specifically due to a CVE that was raised, because people that know way more about security that either of us showed it was too trivially useable for a phishing attack - it will never be '*' by default.

-- TWiki:Main.SvenDowideit - 18 Sep 2007

We need more opinions here.

I accept the attacks fine, and I will happily write for the 5th time that I prefer to handle them in the webserver config. Allow me to explain what I meant: What I meant is that in my config I choose which hosts URIs I accept with a ServerAlias setting - and therefore I (our webserver) do not accept "any SERVER_ADDR".

Whether '*' should be the default or not is probably a question of where we think the balance between intranet (you have webserver control) and extranet (you don't have webserver control) use are at present. I still think we are targeting the intranet primarily.

But hey! personally I can live just fine with '*' not being the default - as long as I can set it to '*' (which I currently can't).

-- TWiki:Main.SteffenPoulsen - 18 Sep 2007

Ok, that i can work with. I was under the impression that you couls set {AllowRedirectUrl} to true, and thus were able to redirect to anywhere - that'd be the redirect part of the '*'

I'm not totally sure about the DefaultHost name '*' - I also would like that, and we both know we used to have it - and that it was removed (I presume as a simple fix to a security thing, but...).

So, does that mean we're still able to say that the issue that Arthur raised is resolved, but that we need to open a new item to rationalise these 2 settings with a '*' defaulthost?

Our approaches are from opposite sides, but not incompatible - I'd rather assume no apache knowledge on the part of the installer, and then provide toold & settings that those that do know can use to expand their horizons.

-- TWiki:Main.SvenDowideit - 19 Sep 2007

Arthur - can you test and close if its working for you?

-- TWiki:Main.SvenDowideit - 20 Sep 2007

There is a perfect solution for this with the solution Sven provided. I have verified that it works, even with a list of allowed URLs and IP addresses. Good work.

it is just very well hidden and could be documented better.

I will do that and close this.

Steffen - your Apache solution hits round 5 or 6 on the TWiki:Codev.NerdoMeter

-- TWiki:Main.KennethLavrsen - 16 Oct 2007

As long as "*" is a possibility, I can accept any solution that gives a better nerdometer score

-- TWiki:Main.SteffenPoulsen - 16 Oct 2007

I do not think * works. But you can add a list of URLs to {PermittedRedirectHostUrls}. Isn't that good enough? Normally you will have 2 or 3 aliases for a domain. Like lavrsen.dk, www.lavrsen.dk, IP address. Remember that the {PermittedRedirectHostUrls} requires the http:// prefix. That is part of the doc improvement I will make. Or wanted to make. "The bugs will not let me save problem" just cost me the half hour I had to fix this one so that will be some other sunny day.

-- TWiki:Main.KennethLavrsen - 16 Oct 2007

is this a can't see the wood for the trees issue?

The code change I made, was to enable users to specify hostnames other than {DefaultHost} to be redirected to if {AllowRedirectUrl} was false (the default).

What the utility of '*' I really cannot fathom, as it seems to me that the intellegent option in the rare case that you want your twiki to be a base for phishers would be to turn {AllowRedirectUrl} to true??

-- TWiki:Main.SvenDowideit - 17 Oct 2007

I understood Steffen's wish as setting {DefaultHost} = *.lavrsen.dk as an example.

Otherwise {DefaultHost} makes no sense. The original wish was to be able to setup a website where both www.lavrsen.dk and lavrsen.dk are valid URLs for the site. A very common wish.

And that is possible to do by setting {DefaultHost} = lavrsen.dk and {PermittedRedirectHostUrls} = http://www.lavrsen.dk.

And you can also add additional domains and IP addresses to {PermittedRedirectHostUrls} comma separated.

This just has to be documented in configure and I will do that while I am in Canada. I will simply had a note to the {DefaultHost} that additional hosts names can be added to the expert setting {PermittedRedirectHostUrls} and under {PermittedRedirectHostUrls} the help text has to say that the URL is of the format "http://domain.com" ie with the http:// prefix.

So if this is an acceptable solution then we can leave it at that.

-- TWiki:Main.KennethLavrsen - 17 Oct 2007

1) People use all kinds of tricks to get to our TWiki installation, and I really don't want to have to be involved every time a new script comes along with a weird IP-URI from a tunnelled network far away.

2) Again, if at last I want to do URI-based filtering, what needs to be done in the virtual host setup in Apache is more than enough for me. It is just doubled work having to do it in the TWiki configuration also.

Any solution not having a * supported is not an acceptable solution for me.

-- TWiki:Main.SteffenPoulsen - 18 Oct 2007

Steffen, did you read my comment from the 17th? I cannot see how what you are talking about is different from using the existing settings. If I'm wrong, you're going to need to elaborate.

-- TWiki:Main.SvenDowideit - 23 Oct 2007

Yes, I think that would work and more to.

My quest is only to enable our site to be nicely edited using URIs that reaches it, though (URIs allowed in apache config) - not to allow a request to be redirected anywhere.

(Technically you can edit the site already so this is not about edit security (your save happens alright), only the the redirect user experience after the save (the error given is still the above mentioned: Access check on topic failed. Action "redirect": unsafe redirect to : host does not match {DefaultUrlHost} , and is not in {PermittedRedirectHostUrls}"").

-- TWiki:Main.SteffenPoulsen - 23 Oct 2007

I am in doubt where this discussion is leading to. Does the code have to be changed or do I document the use of {PermittedRedirectHostUrls} for alternative URLs as I proposed?

-- TWiki:Main.KennethLavrsen - 25 Oct 2007

As I see it there needs to be a way to match the redirect uri against SERVER_NAME and allow it on that basis - which there isn't currently. A way to add the SERVER_NAME cgi variable to {PermittedRedirectHostUrls} would work fine for me.

We are only two opinions here, I am not sure this discussion leads anywhere until more voices show up.

-- TWiki:Main.SteffenPoulsen - 25 Oct 2007

I suggest that Steffen's talking about a new feature (at minimum because I'd have to check some things, see about other platforms / web servers etc), and that this bug is done bar the docco. I'm not firm that we can't do it in 4.2, I simply don't know if we want to up the testing load right now - What happens on IIS? and on the other web servers out there, like JavaServer etc. Should we dissallow SERVER_NAME if its set to *:80, that sort of stuff.

I recon we should close this bug off, and move this discussion to Codev.

-- TWiki:Main.SvenDowideit - 25 Oct 2007

Why add FUD about SERVER_NAME? SERVER_NAME can never be *:80, it is always set by CGI to the URI coming in. Fwiw, IIS and JavaServer also uses CGI to run TWiki.

As I see it the only new feature here is the implementation of this item. TWiki has always serviced URLs that reached it.

-- TWiki:Main.SteffenPoulsen - 26 Oct 2007

I'm not sure that I'm FUDing - but http://www.google.com/search?client=opera&rls=en&q=spoof+SERVER_NAME&sourceid=opera&ie=utf-8&oe=utf-8 suggests to me that there might be an issue, and that we can't just implement it and hope?

It reads to me like SERVER_NAME is untrusted data in the header sent from the client, not a truth. So again, I suggest that the bug reported by Arthur is fixed, and Steffen's quite reasonable request needs to be treated as a new feature that can be given the time to examine it that it deserves. Right now, I'm working on about 3 other security holes, so I'm ot really eager to add another by accident.

-- TWiki:Main.SvenDowideit - 27 Oct 2007

SERVER_NAME is simply the host name sent by the client (directly in the url), and it is filtered by the apache configuration (only requests allowed by the server configuration is passed). As far as I can tell the issues you link to applies to buffer overflows in the webservers themselfes, if SERVER_NAME is not checked appropriately (for specially constructed URLs).

No magic involved.

Again, I'm not asking for this possibility to be in the default setup - only for it to be possible at all.

-- TWiki:Main.SteffenPoulsen - 27 Oct 2007

The more I think about this one the more I tend to agree with Steffen. If I can reach TWiki with a certain URL then I see no reason why TWiki should not work with this URL. We are not talking about redirecting here. If I setup my client machine to translate yahoo.com to the IP address of a TWiki server (e.g. by hacking the hosts file) then I see no reason why the TWIki software should deny access. It did not do that in earlier versions either. I could access with IP address or with alias or anything. Steffen is not asking for a new feature. This worked fine in 4.1.2 where I could access, edit pages etc etc from any URL that led to my server.

I cannot see people being the target of an attack for phishing here because that would require that the attacker could alter the hosts file of the machine of the victim. And "man-in-the-middle" attackers can and would do much worse things so that is not the issue either.

Having to set the EXPERT setting {PermittedRedirectHostUrls} option to allow access makes no sense to the admin. We are not talking redirecting. We are talking access. The purpose and name of {PermittedRedirectHostUrls} has a different meaning and purpose and using it to enhance a list of allowed domain names for access sounds more like a work-around. In my view whatever URL was used to access should work and also be allowed for redirections. I see no security issue with this. It is the client machine that needs to do the spoofing if something gets spoofed. I cannot see who we are protecting here.

If an admin wants to have multiple server names for the same IP address he needs to setup virtual hosts in his apache or whatever server and then only requests that matches the host name for the virtual server will ever reach TWiki anyway.

I cannot see why an admin will have to define the IP address and the server name with www. in the {PermittedRedirectHostUrls} to enable access via IP address, www.domain.com and domain.com. Something I would consider a standard requirement for most admins. Even on my private amateur server I need will now need to add www.lavrsen.dk, localhost, 192.168.1.2 and 62.243.208.118 in additional to the default server name lavrsen.dk. This makes no sense.

I have similar problem at work where we have at least two valid domain names for the same server.

This needs to get fixed.

Steffen why don't you fix the code? It seems to all happen in TWiki.pm.

We need to accept any server name used to access TWiki, and always accept this server name for redirections in addition to {DefaultUrlHost} and {PermittedRedirectHostUrls}. I even think I could write the code myself. But I think you should Steffen since you are a better coder than I.

-- TWiki:Main.KennethLavrsen - 29 Oct 2007

The reason we implemented the {AllowRedirectUrl} == false setting, was to reduce the danger that TWiki is used as a platform to phish other sites.

If you're going to accept anything coming in from a dns / client, (both ways to fake the SERVER_NAME) then you're opening up a nice phishing vector for getting access to other hosts cookies. (thats just one obvious example...)

All that said, I can't see why you don't just change the {AllowRedirectUrl} setting to true. after that , TWiki will allow you to redirect to anywhere, anytime. ZERO code needed.

-- TWiki:Main.SvenDowideit - 29 Oct 2007

Allowing redirects is pretty obvious for phishing purposes, I think I get that. Yet I am not entirely sure why that falls in the same category as allowing a user to be shown a page he has just edited.

-- TWiki:Main.SteffenPoulsen - 29 Oct 2007

Sven. Your feature of {AllowRedirectUrl} is good and right and should be false by default because it is a good protection against phishing. Not a complete protection but part of the defense against phishing.

What Steffen and I want is to add the domain used to access TWiki to the permitted redirect URLs.

So as an example. If I accessed a TWiki with lavrsen.dk and the {DefaultUrlHost} is www.lavrsen.dk, AND the {AllowRedirectUrl} is false and {PermittedRedirectHostUrls} is empty then the two URLs TWiki should allow are: www.lavrsen.dk (because it is the {DefaultUrlHost}) and lavrsen.dk because this is the actual URL used to access the server and the server accepted me.

The reason this becomes a problem is because TWiki does redirects as part its way of working.

I really do not see a security issue in this. I do not believe we should allow * in the {DefaultUrlHost} or allow www in front. I simply think we should silently add the current domain to the list of domains TWiki will redirect to no matter if {AllowRedirectUrl} is true or false and no matter what {PermittedRedirectHostUrls} is defined as. I do not believe we open up any risk doing this. The solution of setting {AllowRedirectUrl} = True however does open up for phishing so that I advice against on a public TWiki. There you have implemented a really good security feature, Sven.

Now there is one thing I want to discuss to be sure I do not miss something here. "Reading the cookies that belongs to other hosts".

Can we walk through this? Sven and Steffen. What are the scenarios and how could accepting redirects to the current URL open up - or not open up attacks?

-- TWiki:Main.KennethLavrsen - 29 Oct 2007

Would hardcoding $TWiki::cfg{DefaultUrlHost} = 'http://' . $ENV{SERVER_NAME}; be a quick way out and overall benefit? So rather than relying on user input, TWiki will rely on the webserver, may it be Apache or lighttpd.

-- TWiki:Main.KwangErnLiew - 31 Oct 2007

wow, thats scary. if you read up, you see should see quite some discussion about the insecurity of SERVER_NAME. Kenneth and I also had a long discussion about the phishing dangers of allowing any servername, rather than limiting it to a pre-determined valid set - gotta get a link to it.

-- TWiki:Main.SvenDowideit - 31 Oct 2007

Sven and I had a very good discussion about this issue on IRC. See http://koala.ilog.fr/twikiirc/bin/irclogger_log/twiki?date=2007-10-29,Mon&sel=291#l287. It is long and runs into next day. The conclusion is that this one is difficult and a trade-off between possible attacks (none has been documented) and usability (which is also a matter of perception)

A small correction of info from above. The domain name from the CGI query->url() is derived from $ENV{'REQUEST_URI'} in CGI. The SERVER_NAME is what Apache thinks it is based on the apache configuration. Above Steffen talks about SERVER_NAME but it is REQUEST_URI he means.

So let us summarize our options

• We can document things better. That is 5 minutes effort. It also involves removing the EXPERT from {PermittedRedirectHostUrls}
• We can still use the doc approach and move the setting up just below the {DefaultUrlHost} so ease the usage a little bit more.
• We can accept the risk that someone can put together an attack vector that takes advantage of spoofing the domain name (this is so easy to do that a small child can do it). The technical implementation could be to leave the default the value of {DefaultUrlHost} to be the domain part of $ENV{'REQUEST_URI'}. Configure defaults to the SERVER_NAME so the default is safe
• We can do nothing and leave things as they are
• Add more if you have ideas

-- KennethLavrsen - 05 Nov 2007

Thanks for clearing up the $ENV{'REQUEST_URI'} part, Kenneth.

Whatever the outcome, we should make sure that this patch is not making things worse for reverse proxy users also (a.la. TWiki:Support.TWikiBehindReverseProxy).

-- TWiki:Main.SteffenPoulsen - 07 Nov 2007

Kenneth - lets start with the docco (1&2) approach, close this Bug for 4.2.0 release, and then start a new one to track the extra work.

-- SvenDowideit - 10 Nov 2007

Yes. Docco approach done. Closing

-- TWiki:Main.KennethLavrsen - 10 Nov 2007

Cleaned "WaitingFor" field.

-- TWiki:Main.GilmarSantosJr - 10 Aug 2008