Gem "devise_token_auth" method: authenticate_user! is returning 401 inspite of valid authentication params

I am trying to authenticate user on every json api request using the gem "devise_token_auth". However, I want to support the legacy devise authentication as well "config.enable_standard_devise_support = true"

#app/controllers/api/v1/api_controller.rb
class Api::V1::ApiController < ApplicationController
  respond_to :json
  include DeviseTokenAuth::Concerns::SetUserByToken
  before_filter :authenticate_user_from_token!
  ....
  def authenticate_user_from_token!
    user_email = request.headers["X-User-Email"].presence || request.headers["uid"].presence
    client_id = request.headers["client"].presence
    access_token = request.headers["access-token"].presence
    user = user_email && User.find_by_email(user_email)
    auth_token = request.headers["X-Auth-Token"].presence
    if user && auth_token && Devise.secure_compare(user.authentication_token, auth_token)
      sign_in user, store: false
      authenticate_user!
    elsif access_token && client_id && user && user.valid_token?(access_token, client_id)
      authenticate_user!
    else
      throw(:warden, scope: :user)
    end
  end
  ....
end

Following is the corresponding rspec

describe "Sign In" do
      it "allows further api requests with valid auth token and denies ones with invalid token" do
        post "/api/v1/sign_in", {"email": @user.email, "password": @user.password}, :format => :json
        expect_status(200)

        auth_header1 = response.header.slice('X-User-Email', 'X-Auth-Token', 'Content-Type')
        auth_header2 = response.header.slice('X-User-Email', 'X-Auth-Token', 'Content-Type')
        auth_header2["X-Auth-Token"] = "WRONGTOKEN"

        # Valid auth token
        get "/api/v1/cycle_days", {}, auth_header1
        expect_status(200)

        # Invalid auth token
        get "/api/v1/cycle_days", {}, auth_header2
        expect_status(401)

        # Cancan failure
        get "/api/v1/user/#{@user.id}", {}, auth_header2
        expect_status(301)
      end
    end

This rspec is getting passed while actual APIs in the postman are failing, after a user has successfully signed in, all of them returning 401 {'error' : 'authentication error'}