EPrints Technical Mailing List Archive

See the EPrints wiki for instructions on how to join this mailing list and related information.

Message: #09627

< Previous (by date) | Next (by date) > | < Previous (in thread) | Next (in thread) > | Messages - Most Recent First | Threads - Most Recent First

Re: [EP-tech] Redirect loop after shibboleth authentication

Hi Lee,

I was finally in a position where I could do some debugging on this and I have got to the bottom of the issue.  Although I still don't think I have fully got my head round why it EPrints session cookie gets sent to the client (i.e. web browser) but then it fails to use it in the next request.  I thought it might be that the server side redirect prevents the cookie from being properly stored and therefore ready for use in the /cgi/users/home request but that would be an issue whatever SameSite was set to.  There are a few Stack Overflow questions about this (e.g. [1]) but none really give a satisfying answer.

Anyway, the solution (although it still does not quite feel right) is to do the redirect within the page rather than server side.  I think the best way to do this is with a meta redirect.  I have updated the shibboleth/login script in [2] to do this.  I have also added some checking of the target URL to make sure this is not trying to redirect you somewhere other than the current EPrints repository. However, here is the main change:

 if( defined $user )
 {
   $user->set_value( "last_login", EPrints::Time::get_iso_timestamp() );
   $user->commit;
 
   EPrints::DataObj::LoginTicket->expire_all( $session );
   $session->dataset( "loginticket" )->create_dataobj({
     userid => $user->id,
   })->set_cookies();
 
   $session->redirect( $url );
 }
 else
 {
   $session->redirect( $session->get_repository->get_conf( "base_dir" ) . "/account_required.html");
 }
 
 $session->terminate;

Becomes:


if( defined $user ) {
  $user->set_value( "last_login", EPrints::Time::get_iso_timestamp() );
  $user->commit;

  EPrints::DataObj::LoginTicket->expire_all( $session );
  $session->dataset( "loginticket" )->create_dataobj({
    userid => $user->id,
  })->set_cookies();
}
else {
  $url = "" "base_dir" ) . "/account_required.html";
}

print '<html><head><meta http-equiv="refresh" content="0;url=''"></head><body</body></html>';
$session->terminate;

Regards

David Newman

[1] https://stackoverflow.com/questions/42216700/how-can-i-redirect-after-oauth2-with-samesite-strict-and-still-get-my-cookies

[2] https://wiki.eprints.org/w/Shibboleth#Configuring_Apache_and_EPrints

On 26/01/2024 14:36, David R Newman wrote:

Hi Lee,

I have been setting up a test environment for this more generally.  However, setting up a Shibboleth IdP is non-trivial, so I am yet to a point where I can test this properly and understand why "SameSite: Strict" is not OK when the redirect that leads to the loop is between two pages both on the same hostname and using the HTTPS (rather than HTTP protocol).  I have wanted to setup a Shibboleth IdP for testing for a while, as I often have to deal with Shibboleth integration issue. It would therefore be nice to gain a greater understanding of how a Shibboleth IdP works, even if there may be some variation between different Shibboleth IdP softwares.

Once I have done some testing I will get back to you.  Using "SameSite: None" is a reasonable solution for now.  It was how EPrints was configured until recently.  However, we decided that security should be enhanced and found no issues with this change when using local authentication.  Having a Shibboleth IdP to test against prior to new EPrints software releases will be of significant benefit, so these sort of things can be picked up earlier or at least we can be sure whether they are a generic issue or specific to a particular EPrints repository configuration.

Regards

David Newman

On 26/01/2024 14:05, Lee Paton wrote:
You don't often get email from patonl@cardiff.ac.uk. Learn why this is important
CAUTION: This e-mail originated outside the University of Southampton.

Hi David

 

Apologies for the slow reply, it looks like my one from last week didn’t appear on the list:

 

I dumped out the following values:

$repository->current_url is set to /cgi/users/home

$repository->current_url( path => "cgi", "users/login" ) is /cgi/users/login

$repository->get_full_url which is https://orca-dev2.cardiff.ac.uk/cgi/users/home


I did find that setting SameSite from “Strict” to “None” in perl_lib/EPrints/DataObj/LoginTicket.pm allowed me to login successfully.

There is a difference between the test idp I used for this dev service and our production idp, the production idp is behind a load balancer and has a rule that sets SameSite=None on requests. It may be this that’s stripping it out, I’m in the process of getting the metadata added to our production idp and authenticating against that to see if this is the difference

Once again, thanks for your help

 

Lee

 

 

From: David R Newman <drn@ecs.soton.ac.uk>
Date: Tuesday, 23 January 2024 at 21:53
To: eprints-tech@ecs.soton.ac.uk <eprints-tech@ecs.soton.ac.uk>, Lee Paton <PatonL@cardiff.ac.uk>
Subject: Re: [EP-tech] Redirect loop after shibboleth authentication

Hi Lee,

That is odd.  I wonder if this is something to do caused as a result of the back and forth for Shibboleth login, as I don't see the same issue with using local login.  In theory the secure_eprints_session cookie would be dropped if you go off site (due to SameSite Strict).  However, it is only created once you have returned from your Shibboleth authentication site and then the redirect between the request that generates this cookie and the page that it loops with are both on your EPrints repository so that should be OK.  I will need to see if I can configure an test EPrints instance with Shibboleth login and see whether this leads to a redirect loop.

If you still were having issues I would have asked if you had any local login account you could have test by going to  https://orca-dev2.cardiff.ac.uk/cgi/users/login and see if you get stuck in a redirect loop after you submit you username and password for the login form on that page.

Regards

David Newman

On 19/01/2024 4:21 pm, Lee Paton wrote:

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Thanks John

 

$repository->current_url is set to /cgi/users/home

$repository->current_url( path => "cgi", "users/login" ) is /cgi/users/login

 

 

I’ve also dumped out $repository->get_full_url which is https://orca-dev2.cardiff.ac.uk/cgi/users/home

 

I’ve changed the SameSite setting from “Strict” to “None” in perl_lib/EPrints/DataObj/LoginTicket.pm and logins are now successful – I have no idea why it’s “Strict” isn’t being set on the CentOS 7 server, most likely an Apache “thing”

 

Lee

 

 

 

From: eprints-tech-request@ecs.soton.ac.uk <eprints-tech-request@ecs.soton.ac.uk> on behalf of John Salter <J.Salter@leeds.ac.uk>
Date: Friday, 19 January 2024 at 15:21
To: eprints-tech@ecs.soton.ac.uk <eprints-tech@ecs.soton.ac.uk>
Subject: RE: [EP-tech] Redirect loop after shibboleth authentication

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

It might be worth dumping out these two before that block, to see what the difference is.

$repository->current_url

$repository->current_url( path => "cgi", "users/login" )

It could be that one (internally) is using http and the other https.

 

NB Things like HSTS can cause additional confusion.

 

Cheers,
John

 

From: eprints-tech-request@ecs.soton.ac.uk <eprints-tech-request@ecs.soton.ac.uk> On Behalf Of Lee Paton
Sent: 19 January 2024 14:54
To: eprints-tech@ecs.soton.ac.uk
Subject: Re: [EP-tech] Redirect loop after shibboleth authentication

 

CAUTION: External Message. Use caution opening links and attachments.


CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi Dave/John

 

First up, John, apologies, I copied and pasted the cookie info from the incorrect site, the ones for orca-dev2.cardiff.ac.uk are:

 

{

        "Request Cookies": {

                "_ga": "GA1.1.996006989.1705664539",

                "_ga_RRV66RFBKP": "GS1.1.1705664538.1.1.1705664547.0.0.0",

                "_shibsession_64656661756c7468747470733a2f2f6f7263612d646576322e636172646966662e61632e756b": "_2246c5adca3fd0bb76b0dd606314177a",

                "secure_eprints_session:orca-dev2.cardiff.ac.uk": "e77e7957efa0d20f1dad5c99b7b429d6"

        }

}

 

 

I’ve put some extra logging into perl_lib/EPrints/Apache/Auth.pm and found that the piece of code it doesn’t get past is

 

                if( $repository->current_url ne $repository->current_url( path => "cgi", "users/login" ) )

                {

                        EPrints::Apache::AnApache::send_status_line( $r, 302, "Need to login first" );

                        EPrints::Apache::AnApache::header_out( $r, "Location", $login_url );

                        EPrints::Apache::AnApache::send_http_header( $r );

                        return DONE;

                }

 

$login_url is set to https://orca-dev2.cardiff.ac.uk/shibboleth/login?target=https%3A%2F%2Forca-dev2.cardiff.ac.uk%2Fcgi%2Fusers%2Fhome here

 

It calls the auth_cookie routine continuously as

 

my $user = $repository->current_user;

 

is never defined at the beginning, on the CentOS 7 system this code is run once, and the next time the auth_cookie routine is run $repository->current_user is defined

 

Our apache builds do change in time so it’s likely that a setting has been changed, I’ll check what’s configured to see if anything shows up

 

Thanks again

 

Lee

 

 

From: David R Newman <drn@ecs.soton.ac.uk>
Date: Friday, 19 January 2024 at 12:34
To: eprints-tech@ecs.soton.ac.uk <eprints-tech@ecs.soton.ac.uk>, Lee Paton <PatonL@cardiff.ac.uk>
Subject: Re: [EP-tech] Redirect loop after shibboleth authentication

Hi Lee,

For 3.4.5 there was a change from 3.4.4 which set SameSite to Strict rather than None, as there were concerns over security using the previous setting [1].  However, I interpreted that you said that both your current CentOS 7 and your upgraded RHEL 8 server were both running EPrints 3.4.5 so this would not have changed.  

Another change between 3.4.4 and 3.4.5 is that if you have "securehost" configured then only the "secure_eprints_session..." cookie will be set and not the "eprints_session cookie" [2].  This is why I was concerned that at some point in your loop you may have hit an HTTP rather than HTTPS page for /cgi/users/home and therefore there would be no session cookie to make /cgi/users/home pick up you were already authenticated, as there would be no "eprints_session cookie" is this would have been disabled because you have "securehost" configured and there would be no "secure_eprints_session" sent because the URL would were request was not secure (i.e. HTTPS) and your browser will not send the "secure_eprints_session" cookie in a request if the "secure" flag is set to "true",

My advice would be to check your /shibboleth/login script check to see what is set as the parameter (e.g. $url ) for $session->redirect( $url ).  If this uses HTTP rather than HTTPS that will be your problem.  Looking at the wiki page [3] the default value for URL is the "userhome" config variable but it should use the target parameter if that is set.  From your example it looks like this is an HTTPS URL, so maybe this is not the issue but I cannot think of anything else it could be based on the information you have so far provided.

Regards

David Newman

[1] https://github.com/eprints/eprints3.4/issues/311

[2] https://github.com/eprints/eprints3.4/issues/278

[3] https://wiki.eprints.org/w/Shibboleth#EPrints_.2Fshibboleth.2Flogin_script_with_user_account_creation

On 19/01/2024 12:07, Lee Paton wrote:

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi Dave

 

Thanks for your quick reply

 

This is what I’m seeing getting the cookie info from Firefox from the request

 

./shibboleth/login?target=https%3A%2F%2Forca-dev2.cardiff.ac.uk%2Fcgi%2Fusers%2Fhome

 

{

        "Response Cookies": {

                "secure_eprints_session:orca-dev2.cardiff.ac.uk": {

                        "domain": "orca-dev2.cardiff.ac.uk",

                        "path": "/",

                        "samesite": "Strict",

                        "secure": true,

                        "value": "1d48249d81423fe7fcd6e37d8b00a748"

                }

        }

}

 

{

        "Request Cookies": {

                "_ga": "GA1.1.512524483.1705663780",

                "_ga_RRV66RFBKP": "GS1.1.1705663779.1.0.1705663784.0.0.0",

                "_shibsession_xxx": "_9e2b8035712cbd57a6b7dc816a7545e4"

        }

}

 

And when it redirects to ./cgi/users/home (where it loops)

 

{

        "Request Cookies": {

                "_ga": "GA1.1.512524483.1705663780",

                "_ga_RRV66RFBKP": "GS1.1.1705663779.1.0.1705663784.0.0.0",

                "_shibsession_xxx": "_9e2b8035712cbd57a6b7dc816a7545e4"

        }

}

 

When I refresh the cookies are

 

{

        "Request Cookies": {

                "_ga": "GA1.1.1118556027.1705664127",

                "_ga_RRV66RFBKP": "GS1.1.1705664127.1.0.1705664127.0.0.0",

                "_shibsession_xxx": "_27e34d6d248703d1d0cacdc14ad393f8",

                "secure_eprints_session:orca-dev.cardiff.ac.uk": "1a81565f2d54425ba0a6ed135d7feb3c"

        }       

}    

 

The difference between this and the working CentOS 7 server is that the secure_eprints_session value is set on the initial request to /cgi/users/home, I’ve also noticed that

 

"samesite": "Strict",

 

Isn’t set on the CentOS 7 server – As far as I’m aware I haven’t changed this at all so should be set to strict on both

 

Thanks again

 

Lee

 

From: David R Newman <drn@ecs.soton.ac.uk>
Date: Thursday, 18 January 2024 at 15:18
To: eprints-tech@ecs.soton.ac.uk <eprints-tech@ecs.soton.ac.uk>, Lee Paton <PatonL@cardiff.ac.uk>
Subject: Re: [EP-tech] Redirect loop after shibboleth authentication

External email to Cardiff University - Take care when replying/opening attachments or links.

Nid ebost mewnol o Brifysgol Caerdydd yw hwn - Cymerwch ofal wrth ateb/agor atodiadau neu ddolenni.

 

Hi Lee,

So there could be a few issues here.  It may be useful if you can show a couple of iterations of the redirect loop, as this may help identify some minor technicality like a switch between HTTP and HTTPS or a missing or present trailing slash.

/shibboleth/login should redirect you to /cgi/users/home if you current user session has appropriate environment variables set for an authenticated Shibboleth session, otherwise is will like redirect you to /account_required.html.  It seems like /shibboleth/login is redirecting to /cgi/users/home and then as it cannot find the secure_eprints_session... cookie it is redirecting back to /shibboleth/login,   Clearly this could be a race condition with the secure_eprint_session not being set and therefore sent a part of your request for /cgi/users/home. However, when you manually refresh the page your user session has received the required cookie and then sends it in this request.

From what you have said your version of EPrints has not changed (it was 3.4.5 prior to the migration) and all that has really changed is a newer operating system, with virtually everything else much the same.  My suspicion is that at some point there is an HTTP rather than HTTPS URL, which means the secure_eprints_session will not be available as it has "Secure" attribute set to "true".  When you do see secure_eprints_session cookie it would be useful if you could share the its attribute like my example below:

secure_eprints_session%3Aexample.eprints.org:"ef9def123af12384e9abcd1a49ea27abc"
Created:"Wed, 17 Jan 2024 14:45:37 GMT"
Domain:".example.eprints.org"
Expires / Max-Age:"Session"
HostOnly:false
HttpOnly:false
Last Accessed:"Thu, 18 Jan 2024 15:12:39 GMT"
Path:"/"
SameSite:"Strict"
Secure:true
Size:82

 

Regards

 

David Newman

 

On 18/01/2024 14:46, Lee Paton wrote:

CAUTION: This e-mail originated outside the University of Southampton.

CAUTION: This e-mail originated outside the University of Southampton.

Hi

 

I’m currently migrating our eprints (version 3.4.5) repository from CentOS 7 to RHEL 8, everything has gone smoothly  apart from the repository going into a redirect loop after the session has authenticated and you’re redirected back to a logged in session on the repo with:

 

https://orca-dev2.cardiff.ac.uk/shibboleth/login?target=https%3A%2F%2Forca-dev2.cardiff.ac.uk%2Fcgi%2Fusers%2Fhome

 

If you then hit return on the browsers address the users home page is displayed successfully

 

Looking at the http headers it appears that the secure_eprints_session value isn’t being set in a cookie (I’ve compared headers to the working CentOS 7 dev system) on that first redirect after shibboleth authentication has completed

 

The config is identical to the one that is currently running on our CentOS 7 environment, and the versions of Apache are also the same, there is a small difference in perl versions (5.16 and 5.26). The main difference is that I’m using our test idp to authenticate against using a hosts file change on my macbook instead of using out production idp.  Would this make a difference?

 

Any pointers or suggestions would be much appreciated – I’ve hit a brick wall with this

 

Thanks

 

Lee

 

*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List
*** Archive: htt
 ps://www.eprints.org/tech.php/
*** EPrints community wiki: https://wiki.eprin
 ts.org/
 

 

*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List
*** Archive: https://www.eprints
 .org/tech.php/
*** EPrints community wiki: https://wiki.eprints.org/<
 /pre>
                                              
 

                                            



*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List
*** Archive: https://www.eprints.org/tech.php/
*** EPrints community wiki: https://wiki.eprints.org/
 

 


*** Options: https://wiki.eprints.org/w/Eprints-tech_Mailing_List
*** Archive: https://www.eprints.org/tech.php/
*** EPrints community wiki: https://wiki.eprints.org/