Login to site using CSRF token using Go/Python

I want to backup automatically web content from a site which requires login. I try to login by simulating a POST request. But I get the error:

csrf token: CSRF attack detected

Here are some extracts from the code I use:

func postLoginForm(csrfToken string) {
	values := make(url.Values)
    values.Set("signin[username]", "myusername") 
    values.Set("signin
		

			
输入密码查看隐藏内容
			

				
				
			
		
", "mypassword") 
	                                   
	values.Set("signin[_csrf_token]", csrfToken)
	resp, err := http.PostForm("https://spwebservicebm.reaktor.no/admin/nb", values)
	
	dumpHTTPResponse(resp) // show response to STDOUT
}

The csrf token I get by fetching the login page and scanning it for a hidden input field named signin[_csrf_token]. The important part of the code for doing that is the following:

// Finds input field named signin[_csrf_token] and returns value as csrfToken
func handleNode(n *html.Node) (csrfToken string, found bool) {
	if n.Type == html.ElementNode && n.Data == "input" {
		m := make(map[string]string)
		for _, attr := range n.Attr {
			m[attr.Key] = attr.Val
		}
		if m["name"] == "signin[_csrf_token]" {
			return  m["value"], true
		}
	}
	
	for c := n.FirstChild; c != nil; c = c.NextSibling {
		 if csrfToken, found = handleNode(c); found {
			 return 
		 }		 
	}
	
	return "", false
}

I don't need to be using Go, that is just because I am most familiar with that. Using python could be a solution as well, but I did not have any more luck with that.

The issue is that Go 1.2 does not automatically use a cookie jar for its HTTP requests. The first request is to get the CSRF token from the login page. The second request is to POST a login using that CSRF token. But since no session cookie is attached to the HTTP header on the second request the server does not know that it is the same program trying to login. Thus the server thinks it is a CSRF attempt (that you picked the CSRF token from somewhere else and tried to reuse it).

So to get login page and extract CSRF token, we first create our own client object. Otherwise we have nowhere to attach the cookie jar. http.PostForm does give access to cookie jar:

client = &http.Client{}

Create a cookie Jar described in https://stackoverflow.com/questions/11361431/authenticated-http-client-requests-from-golang . This was easier to setup and debug than the official: http://golang.org/pkg/net/http/cookiejar/ Cookie Jar

jar := &myjar{}
jar.jar = make(map[string] []*http.Cookie)
client.Jar = jar	

resp, err := client.Get("https://spwebservicebm.reaktor.no/admin")		
doc, err := html.Parse(resp.Body)

Then to login, we reuse the client object with the cookie jar attached to it:

values := make(url.Values)
values.Set("signin[username]", "myusername")
values.Set("signin
		

			
输入密码查看隐藏内容
			

				
				
			
		
", "mypassword")	                                   
values.Set("signin[_csrf_token]", csrfToken)
	
resp, err := client.PostForm("https://spwebservicebm.reaktor.no/admin/login", values)

You'll notice that the code is almost identical to the one in the question except we use client.PostForm instead of http.PostForm.

Thanks to dommage and answer to https://stackoverflow.com/questions/11361431/authenticated-http-client-requests-from-golang for getting me on right track.

You can scrape the token using beautifulsoup and store it, either in the header or send it to the server. You can do something like this:

from requests import session
from bs4 import BeautifulSoup

def authent(self):
    ld('trying to get token...')
    r = self.session.get(BASE_URL, headers=FF_USER_AGENT)
    soup = BeautifulSoup(r.content)
    elgg_token = soup.select('input[name="__elgg_token"]')[0]["value"]
    elg_ts = soup.select('input[name="__elgg_ts"]')[0]["value"]
    payload["__elgg_token"] = elgg_token # I sent it to the server...
    payload["__elgg_ts"] = elg_ts
    r = self.session.post(LOGIN_URL, data=payload, headers=FF_USER_AGENT)
    if r.url != DASHBOARD_URL:
      raise AuthentError("Error")