(define ax+by=1
(lambda (a b)
(if (= a 0)
(list 0 1)
(let ((pair (ax+by=1 (remainder b a) a)))
(cons (- (cadr pair)
(* (quotient b a) (car pair)))
(list (car pair)))))))
; > (ax+by=1 17 13)
; '(-3 4)
Your "problem" was that your function was returning a cons-cell whose last element is NOT '().
A list is any chain of cons-cells with the last element as nil.
If your code does (cons 3 4), it returns (3 . 4). Or in your notation (it is not Racket, isn't it? Or a subdialect of Racket? (mcons 3 4).
To make it a normal list, it has to become (cons 3 (cons 4 '())) which is exactly the same like (list 3 4). list takes the arguments and conses them on '() in its last cons cell.
(define ax+by=1
(lambda (a b)
(if (= a 0)
(cons 0 (cons 1 '()))
(let ((pair (ax+by=1 (remainder b a) a)))
(cons (- (cadr pair)
(* (quotient b a) (car pair)))
(cons (car pair) '()))))))
Super lazy solution
Or, let's say, you are super lazy, and you see that your old function actually does what it should - only the output format hast to be changed from pair to list.
With this function, you could transform the pair to a list:
(define (pair->list p)
(list (car p) (cdr p))) ; just list the components of pair
You could then take your old function - unmodified - then the transformer function and wrap around it a function definition - even with the same name (however, this is REALLY BAD for readability! - however, interesting, that this works in Racket ...).
(define ax+by=1
(lambda (a b)
;; your old function verbatim as inner `ax+by=1`:
(define ax+by=1
(lambda (a b)
(if (= a 0)
(cons 0 1)
(let ((pair (ax+by=1 (remainder b a) a)))
(cons (- (cdr pair)
(* (quotient b a) (car pair)))
(car pair))))))
;; the transformer function verbatim:
(define (pair->list p)
(list (car p) (cdr p)))
;; call both inner functions in combination!
(pair->list (ax+by=1 a b))))
;; `pair->list` transforms your answer into the correct list form!
Notable of this last version is, that in the last function call of ax+by=1, the interpreter/compiler "knows" that the inner function is meant and not the outer function (else the outer function would call itself again and again passing the given arguments in an endless loop). This is possible, because the inner function-name-binding "shadows" the outer function-name-binding.
However, I would consider it really bad style, because the human reader might get very confused with the identical name between inner and outer function.
Nevertheless, I found it interesting to find a way to not to modify existing code and just add something to make the thing work and even use the intended old name - so that the result is exactly the desired result.
But definitely the first solutions I gave are better - no possibility to misunderstand the code and confuse things - and direct construction of the results-list.
