How create a middleware with api endpoints in .NET Core

I have created the web application with the web api. The application contains some Controllers for example TodoController:

namespace TodoApi.Controllers
{
    [Route("api/[controller]")]
    public class TodoController : Controller
    {
        private readonly TodoContext _context;

        public TodoController(TodoContext context)
        {
            _context = context;
        }       

        [HttpGet]
        public IEnumerable<TodoItem> GetAll()
        {
            return _context.TodoItems.ToList();
        }
    }
}

If I create the GET request - /api/todo - I get the list of Todos from database.

I have a list of controllers and api endpoints like above.

I would like distribute this api to another application ideally like middleware - my idea is register in Startup.cs like this:

public void ConfigureServices(IServiceCollection services)
{
  services.AddTodoApi();
}

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
  app.UseTodoApi();
}

This will be awesome use case for my api but I don't know how this controllers api endpoints rewrite like middleware and return same JSON data same approache like using classic Controllers.

How can I write the middleware in .NET Core for creating API endpoints?