Weird malformed cookies returned from REST api (sometimes)

user_685268 0
Viewed 1378

I have an interesting problem - sometimes malformed cookies are returned from my web API hosted on AWS (.NET Core 2.2), behind Incapsula WAF and AWS elb. This messes up my .NET framework SDK which throws exception when trying to parse headers complaining about server doing protocol violation (suggestions are usually this: https://weblog.west-wind.com/posts/2007/Mar/29/The-server-committed-a-protocol-violation-with-WebRequest - setting useUnsafeHeaderParsing to true).

In fiddler, I managed to figure out the problem (likely): Fiddler screenshot

Anyone knows where these cookie headers come from? (as you can see some headers are added by AWS/WAF), but they don't seem to be the problem. Important: those __utm* cookies are only set sometimes. Googling it, only points to possible GA cookies, but they seem different (already checked https://www.webtrafficexchange.com/google-analytics-cookies-utma-utmb-utmz)

asp.net-core amazon-web-services linux
2 Answers

See this reply we got from Imperva support regarding this Incapsula issue:

The Imperva Cloud WAF injects cookies in client requests, purposely malformed, as part of the client classification process. This is known as the classification cookie or testing cookie. This allows the WAF to see if the client can parse that purposely malformed cookie to determine the client making the request, i.e. Bot or Human (browser) all modern browsers can parse the cookie. This is known to cause issues with APIs & we can disable this on your request. Would you like us to disable this cookie? Please confirm. Its disabled by Imperva admins only, via a back-end configuration.

I believe it should resolve your issue.

Edit for additional info/context (following PST's comment):

Imperva blog post on their anti-bots client classification process: https://www.imperva.com/blog/how-incapsula-client-classification-challenges-bots/

The entire process is based on using challenges where bots are expected to react differently than regular browsers or humans.

Specifically on the topic of the original issue with cookies:

It seems Incapsula first used plain cookies based on the fact that most bots did not return cookies to the server at all. Bots than got smarter and many of them now do return the cookies. So the process was improved to include a malformed cookie that regular browsers discard (allowing Incapsula to again tell them apart from bots). So now they seem to be sending both types of cookies (as can be seen in the original post) for the full solution: A regular one to see if the client supports cookies at all. A malformed one to still tell apart bots even if they do support cookies.

sources:

  1. https://github.com/postmanlabs/newman/issues/2747
  2. https://www.neptuneenergy.com/cookie-policy (quoted below in case it changes)

__utmv Incapsula Classification cookie: To see how the client reacts and handles a malformed cookie in order to identify what that client is. lives up to 900 seconds

user_16802810

From https://www.google.com/amp/s/www.morevisibility.com/blogs/analytics/from-utma-to-utmz-google-analytics-cookies.html/amp/

The __utmv Cookie If you are making use of the user-defined report in Google Analytics, and have coded something on your site for some custom segmentation, the __utmv cookie gets set on the person’s computer, so that Google Analytics knows how to classify that visitor. The __utmv cookie is also a persistent, lifetime cookie.

It definitely appears to be a Google Analytics tracking cookie. It must be embedded somewhere. Not sure what part of your stack would add those.

You could consider overriding them or removing them to prevent the parsing error. See https://aws.amazon.com/blogs/compute/overriding-request-response-parameters-and-response-status-in-amazon-api-gateway/

user_54514920
Related