Apologies for a rather lengthy question, but I don't think I could leave any of it out and still have a minimum reproducible example.
The problem
My web application stores the user's name in a cookie. If the cookie is not found when the user visits the application then the application prompts the user to enter their name and then stores that name in a cookie.
If the cookie is found when the user visits the application then the application greets the user by the name stored in the cookie.
If the user decides they want the application to forget them then the application deletes the cookie, and the next time the user visits the application, the application prompts the user to enter their name.
When I manually test the application, everything appears to be working fine. However, when I use Selenium to run a test scenario where the user has already has such a cookie, to validate that the application greets them by name, the application behaves as if the cookie doesn't exist, and instead of greeting the user by name it prompts them to enter their name.
I believe the "Arrange" part of my test scenario is not setting the cookie up correctly, however I can't see why.
Tools and technologies
The application consists of a React.js client and an ASP.net MVC core web API server written in C#.
The Selenium test project uses NUnit as a test runner and is written in C#.
Both the Selenium test project and the web API server target version 3.1 of .net core.
The server
The web API server project has a single controller, listening to the /api/User route, with 3 action methods:
- GetCurrent - responds to GET requests, looks for the UserName cookie, returns the user name in that cookie if it exists and returns a 404 not found status if it doesn't exist.
- New - responds to POST requests, expects a userName parameter in the query string, and creates a UserName cookie containing that user name if it's valid.
- Forget - responds to DELETE requests and deletes the UserName cookie.
UserController.cs
This is the only controller in the project.
namespace CookieServer.Controllers
{
using System;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
[Route("api/[controller]")]
[ApiController]
public class UserController : Controller
{
/// <summary>
/// Gets the name of the current user from the UserName cookie, if it exists.
/// </summary>
/// <returns>200 with the user's name if the cookie exists, otherwise 404.</returns>
[HttpGet]
public IActionResult GetCurrent()
{
string userName;
var cookieExists = this.HttpContext.Request.Cookies.TryGetValue("UserName", out userName);
Console.WriteLine($"Cookie exists: {cookieExists}");
if (cookieExists)
{
return this.Ok(new { UserName = userName }); // 200
}
else
{
return this.NotFound(); // 404
}
}
/// <summary>
/// Creates a cookie called "UserName" with the value set to the supplied userName parameter.
/// If the user later visits the site from the same client machine and browser, we'll remember their name.
/// </summary>
/// <param name="userName">The username to store in the cookie.</param>
/// <returns>200 if the username is OK, otherwise 400.</returns>
[HttpPost]
public IActionResult New(string userName)
{
if (string.IsNullOrWhiteSpace(userName))
{
return this.ValidationProblem("userName parameter must not be null or whitespace"); // 400
}
var options = new CookieOptions
{
// Set SameSite to Lax because the React.js client and this web API service are running
// on different ports, and so count as different domains.
SameSite = SameSiteMode.Lax,
Secure = true, // Only send the cookie over HTTPS
// Allow client-side JavaScript to access the cookie.
// Normally we'd set this to true to prevent scripting attacks, but I want the client to
// be able to log this cookie, otherwise the Selenium tests have no way of telling
// whether or not the client has this cookie.
HttpOnly = false,
// The cookie will expire in about 8 thousand years.
// This isn't quite the same as never expiring, but I'm pretty sure .net core 3.1 will be
// out of support by then ;-)
Expires = DateTime.MaxValue,
};
this.HttpContext.Response.Cookies.Append("UserName", userName, options);
return this.Ok(); // 200
}
/// <summary>
/// Deletes the UserName cookie, and so effectively forgets the user.
/// </summary>
/// <returns>200.</returns>
[HttpDelete]
public IActionResult Forget()
{
this.HttpContext.Response.Cookies.Delete("UserName");
return this.Ok(); // 200
}
}
}
Startup.cs
namespace CookieServer
{
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Net.Http.Headers;
public class Startup
{
public Startup(IConfiguration configuration)
{
Configuration = configuration;
}
public IConfiguration Configuration { get; }
/// <summary>
/// This method gets called by the runtime. Use this method to add services to the container.
/// </summary>
/// <param name="services">Service collection</param>
public void ConfigureServices(IServiceCollection services)
{
services.AddCors(options =>
options.AddPolicy("Dev", builder =>
builder.WithMethods("GET", "POST", "PUT", "DELETE")
.WithHeaders(
HeaderNames.Accept,
HeaderNames.ContentType,
HeaderNames.Authorization)
.AllowCredentials()
.SetIsOriginAllowed(origin =>
{
if (string.IsNullOrWhiteSpace(origin)) { return false; }
// Remove the next line in production
if (origin.ToLower().StartsWith("http://localhost")) { return true; }
return false;
})));
services.AddControllers();
}
/// <summary>
/// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
/// </summary>
/// <param name="app">Application builder.</param>
/// <param name="env">Web host environment.</param>
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
app.UseHttpsRedirection();
app.UseCors("Dev");
app.UseRouting();
app.UseAuthorization();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
});
}
}
}
The client
The React.js application which acts as the client was created using create-react-app, which configures Babel for me so that I can use ES6 language features such as classes and the fetch API without having to worry about whether the user's browser supports them.
MyComponent.jsx
This is the only component I've added to the application created by create-react-app. It makes a GET request to the server to find out whether the UserName cookie exists, and based on the response, it either prompts the user to enter their name (and makes a POST request to the server to create the UserName cookie), or it greets the user by name and gives them a "forget me" button (which makes a DELETE request to the server to delete the UserName cookie).
import React from 'react';
export default class MyComponent extends React.Component {
/**
* Initializes a new instance of the MyComponent class.
* @param {object} props values passed to the component as JSX attributes.
*/
constructor(props) {
super(props);
this.state = {
checkingUserName: false,
userName: '',
apiError: '',
enteredUserName: '',
};
// The port number in this URL is taken from the "sslPort" key
// of the launchSettings.json file in the web API project.
this.apiUrl = 'https://localhost:44358/api/User';
}
/**
* Called by the React runtime immediately after the component is mounted.
* In this component we want to call the web API to see whether it knows
* who the user is.
*/
componentDidMount = () => {
this.checkWhoIAm();
}
/**
* Called by the React runtime when the component's state changes.
* @returns {string} JSX markup representing the component.
*/
render = () => {
const nameTextBox = (
<input
type="text"
id="nameTextBox"
placeholder="e.g. John Doe"
onChange={(e) => this.setState({enteredUserName: e.target.value})}
/>
);
const rememberMeButton = (
<button
type="button"
id="rememberMeButton"
onClick={this.rememberMe}
>
Remember me
</button>
);
const forgetMeButton = (
<button
type="button"
id="forgetMeButton"
onClick={this.forgetMe}
>
Forget me
</button>
);
const apiError = this.state.apiError ? <div>{this.state.apiError}</div> : '';
let jsx;
if (this.state.checkingUserName) {
jsx = <div>Checking who you are...</div>;
} else if (this.state.userName) {
jsx = (
<div>
{apiError}
<div id="welcome">Welcome, {this.state.userName}</div>
{forgetMeButton}
</div>
);
} else {
jsx = (
<div>
{apiError}
<div>Please enter your name:</div>
{nameTextBox}
{rememberMeButton}
</div>
);
}
return jsx;
}
/**
* Makes a GET request to the web API to see whether there's
* already a UserName cookie, and if so stores the user's
* name in the component state.
*/
checkWhoIAm = async () => {
const options = {
headers: {},
method: 'GET',
credentials: 'include',
mode: 'cors',
};
this.setState({
checkingUserName: true,
userName: '',
apiError: '',
});
try {
console.log("GET");
const response = await fetch(this.apiUrl, options);
if (response.status === 200) {
console.log(`Cookies: ${document.cookie}`);
this.setState({
checkingUserName: false,
userName: (await response.json()).userName,
apiError: '',
})
} else {
console.log(`Cookies: ${document.cookie}`);
this.setState({
checkingUserName: false,
userName: '',
});
}
} catch (exception) {
this.setState({
checkingUserName: false,
userName: '',
apiError: exception.message,
});
}
}
/**
* Makes a POST request to the web API to store the name
* the user entered in a UserName cookie.
*/
rememberMe = async () => {
const options = {
headers: {'Content-Type': 'application/json'},
body: {},
method: 'POST',
credentials: 'include',
mode: 'cors',
};
try {
console.log("POST");
const url = `${this.apiUrl}?userName=${this.state.enteredUserName}`;
const response = await fetch(url, options);
if (response.status === 200) {
console.log(`Cookies: ${document.cookie}`);
this.setState({
userName: this.state.enteredUserName,
apiError: '',
});
} else {
this.setState({apiError: JSON.stringify(await response.json())});
}
} catch (exception) {
this.setState({apiError: exception.message});
}
}
/**
* Makes a DELETE request to the web API to delete the UserName cookie.
* The user has a right to be forgotten!
*/
forgetMe = async () => {
const options = {
method: 'DELETE',
credentials: 'include',
mode: 'cors',
}
try {
console.log("DELETE");
const response = await fetch(this.apiUrl, options);
if (response.status === 200) {
console.log(`Cookies: ${document.cookie}`);
this.setState({
userName: '',
apiError: '',
});
} else {
this.setState({apiError: JSON.stringify(await response.json())});
}
} catch (exception) {
this.setState({apiError: exception.message});
}
}
}
App.js
This file is created by create-react-app, and by default renders the React logo. I changed it to render MyComponent instead.
import './App.css';
import MyComponent from './MyComponent';
function App() {
return (
<MyComponent />
);
}
export default App;
The Selenium test
The test attempts to create a UserName cookie, navigates to the application's home page, and asserts that the element with the ID "welcome" contains the user name in the cookie. It then captures what was written to the browser's console so that I can see the requests the UI is making to the server, and the cookies that the UI thinks it has.
namespace SeleniumCookiesUITest
{
using System;
using System.Linq;
using System.Net.Http;
using System.Threading.Tasks;
using NUnit.Framework;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support.UI;
[TestFixture]
public class SeleniumCookieTests
{
/// <summary>
/// URL of the server-side API service.
/// </summary>
private readonly string _apiUrl = "https://localhost:44358/api/User/";
/// <summary>
/// URL of the React.js client.
/// </summary>
private readonly string _uiUrl = "http://localhost:3000";
[Test]
public async Task GivenIAmAnExistingUser_WhenILoadThePage_ThenIWillBeGreetedByName()
{
// Arrange
var options = new ChromeOptions();
options.SetLoggingPreference(LogType.Browser, LogLevel.All);
var driver = new ChromeDriver(options);
try
{
var userName = Guid.NewGuid().ToString();
// Uncomment one of the following lines and comment out the others
SetCookieUsingDriver(driver, userName);
//await SetCookieUsingApi(userName);
//SetCookieUsingUI(driver, userName);
// Act
driver.Navigate().GoToUrl(_uiUrl);
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
// Assert - fails with timeout - 'Unable to locate element: {"method":"css selector","selector":"#welcome"}'
var welcome = wait.Until(e => e.FindElement(By.Id("welcome")));
StringAssert.Contains(userName, welcome.Text);
}
finally
{
var logs = driver.Manage().Logs.GetLog(LogType.Browser).ToList();
if (logs.Count > 0)
{
Console.WriteLine($"The following was written to the Browser log...");
}
foreach (var log in logs)
{
Console.WriteLine($"{log.Timestamp} {log.Level} {log.Message}");
}
driver.Quit();
}
}
/// <summary>
/// Creates a UserName cookie using the method shown at
/// <see href="https://www.selenium.dev/documentation/en/support_packages/working_with_cookies/"/>.
/// </summary>
/// <param name="driver">IWebDriver instance.</param>
/// <param name="userName">User name to store in the cookie.</param>
private void SetCookieUsingDriver(IWebDriver driver, string userName)
{
driver.Navigate().GoToUrl(_uiUrl);
driver.Manage().Cookies.AddCookie(new Cookie("UserName", userName));
// Check the cookie has been created
Assert.AreEqual(userName, driver.Manage().Cookies.GetCookieNamed("UserName").Value);
}
/// <summary>
/// Creates a UserName cookie by making a POST request to the API.
/// </summary>
/// <param name="userName">User name to store in the cookie.</param>
/// <returns>A Task representing the asynchronous operation.</returns>
/// <remarks>
/// This method feels wrong because the cookie doesn't seem to be associated with the driver, although
/// <see href="https://www.selenium.dev/documentation/en/guidelines_and_recommendations/generating_application_state/"/>
/// could be interpreted as suggesting this method.
/// </remarks>
private async Task SetCookieUsingApi(string userName)
{
var client = new HttpClient();
// This POST request will create the cookie
var httpRequest = new HttpRequestMessage(HttpMethod.Post, $"{_apiUrl}?userName={userName}");
var response = await client.SendAsync(httpRequest);
response.EnsureSuccessStatusCode();
// This GET request returns the username set in the POST request to prove that the cookie has been created
httpRequest = new HttpRequestMessage(HttpMethod.Get, _apiUrl);
response = await client.SendAsync(httpRequest);
response.EnsureSuccessStatusCode();
var responseContent = await response.Content.ReadAsStringAsync();
StringAssert.Contains(userName, responseContent);
}
private void SetCookieUsingUI(IWebDriver driver, string userName)
{
driver.Navigate().GoToUrl(_uiUrl);
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
var textBox = wait.Until(e => e.FindElement(By.Id("nameTextBox")));
var button = wait.Until(e => e.FindElement(By.Id("rememberMeButton")));
textBox.Clear();
textBox.SendKeys(userName);
button.Click();
}
}
}
There are 3 different methods I've attempted for creating the cookie. Whichever one I use, the test fails because when I'm expecting the welcome message to be displayed, instead the prompt to enter the user's name is displayed.
SetCookieUsingDriver
I believe this uses the method described on Selenium's working with cookies page, however the application isn't finding the cookie and is instead prompting the user to enter their name, hence the element with ID "welcome" isn't displayed. Console output from the test when using this method:
The following was written to the Browser log...
31/07/2021 12:29:04 Info http://localhost:3000/static/js/vendors~main.chunk.js 37052:14 "[HMR] Waiting for update signal from WDS..."
31/07/2021 12:29:04 Info http://localhost:3000/static/js/main.chunk.js 385:16 "GET"
31/07/2021 12:29:04 Severe https://localhost:44358/api/User - Failed to load resource: the server responded with a status of 404 ()
31/07/2021 12:29:04 Info http://localhost:3000/static/js/main.chunk.js 396:18 "Cookies: "
31/07/2021 12:29:04 Info http://localhost:3000/static/js/vendors~main.chunk.js 37052:14 "[HMR] Waiting for update signal from WDS..."
31/07/2021 12:29:04 Info http://localhost:3000/static/js/main.chunk.js 385:16 "GET"
31/07/2021 12:29:04 Severe https://localhost:44358/api/User - Failed to load resource: the server responded with a status of 404 ()
31/07/2021 12:29:04 Info http://localhost:3000/static/js/main.chunk.js 396:18 "Cookies: UserName=aa6d2d23-0534-4b03-9681-bf6a091f8cec"
The first GET request returns a 404 status, I'm expecting this because that request is only to get the browser onto the correct domain before creating the cookie. On the second GET request it appears that the UI thinks it has a UserName cookie but either it isn't being sent to the server, or the server isn't finding it in the request.
SetCookieUsingApi
Rather than using Selenium to create the cookie, this method makes a POST request to the server to create the cookie before launching the application. On reflection, this method feels wrong because there doesn't seem to be any association between whatever cookie was created and the browser window which is then opened, but I thought I'd try it.
The following was written to the Browser log...
31/07/2021 12:42:31 Info http://localhost:3000/static/js/vendors~main.chunk.js 37052:14 "[HMR] Waiting for update signal from WDS..."
31/07/2021 12:42:31 Info http://localhost:3000/static/js/main.chunk.js 385:16 "GET"
31/07/2021 12:42:31 Severe https://localhost:44358/api/User - Failed to load resource: the server responded with a status of 404 ()
31/07/2021 12:42:31 Info http://localhost:3000/static/js/main.chunk.js 396:18 "Cookies: "
SetCookieUsingUI
This method creates the cookie using the UI, it navigates to the application's home page, enters a name into the text box, clicks the "remember me" button and then navigates to the home page again to test whether the entered name is now displayed. This goes against Selenium's guidelines on generating application state, which state
Selenium should not be used to prepare a test case. All repetitive actions and preparations for a test case, should be done through other methods.
However this method is also the most similar to how the application is used, so it's the most surprising that this method also fails.
The following was written to the Browser log...
31/07/2021 12:50:30 Info http://localhost:3000/static/js/vendors~main.chunk.js 37052:14 "[HMR] Waiting for update signal from WDS..."
31/07/2021 12:50:30 Info http://localhost:3000/static/js/main.chunk.js 385:16 "GET"
31/07/2021 12:50:31 Severe https://localhost:44358/api/User - Failed to load resource: the server responded with a status of 404 ()
31/07/2021 12:50:31 Info http://localhost:3000/static/js/main.chunk.js 396:18 "Cookies: "
31/07/2021 12:50:31 Info http://localhost:3000/static/js/main.chunk.js 423:16 "POST"
31/07/2021 12:50:31 Info http://localhost:3000/static/js/main.chunk.js 428:18 "Cookies: "
31/07/2021 12:50:31 Info http://localhost:3000/static/js/vendors~main.chunk.js 37052:14 "[HMR] Waiting for update signal from WDS..."
31/07/2021 12:50:31 Info http://localhost:3000/static/js/main.chunk.js 385:16 "GET"
31/07/2021 12:50:31 Severe https://localhost:44358/api/User - Failed to load resource: the server responded with a status of 404 ()
31/07/2021 12:50:31 Info http://localhost:3000/static/js/main.chunk.js 396:18 "Cookies: "
This time, the UI doesn't seem to think at any point that it has a UserName cookie.
The question
What am I doing wrong here? The fact that the application behaves exactly as expected when manually tested leads me to believe that my UI and server code is sound, therefore the problem must be in the way the test is creating the cookie. The fact that I'm fairly experienced with React and ASP.net MVC core but only started working with Selenium a few months ago lends weight to this view.
However this is also the first time I've tried working with cookies, so my mind remains open to the possibility that the cookie handling in the server side code isn't up to scratch.