Turning Perceptron into Sigmoid/Relu neuron

I made a single perceptron that has a binary threshold activation function ( x < 0 ? 0 : 1 ) and I'm teaching it (updating weights) by this formula:

WEIGHT + (weightDelta * learningRate) where weight delta is (INPUT to this weight * error).

And it works fine, the SAME formula even works if I'm using a sigmoid function instead of a binary threshold. I don't use derivatives while backpropagation and it still works..WHY? When I try to use other activations like RELU it does not work, or with TANH it works only sometimes.. As a proof of concept, I did a quick demo with single perceptron/neuron learning and playing some Arkanoid here from random data: https://codepen.io/sanchopanza/pen/eYJWygy

learn () {
    this.biasWeight = this.biasWeight + (this.bias * (this.desiredOutput - this.output) * 0.1);
    for (var n = 0; n < this.weights.length; n++) {
        this.weights[n] = this.weights[n] + (this.inputs[n] * (this.desiredOutput - this.output) * 0.1);
    }
}

Even tho sigmoid is working, I think I'm heavily wrong here, and while using non-linear activation functions the backpropagation formula should use DERIVATIVE as weights delta. How do I find a derivative of sigmoid and RELU for example? I tried using formula when weightsDelta is calculated with OUTPUT * (1 - OUTPUT) for sigmoid, where output is already sigmoid and I just try to find its derivative but it just makes it worse, also tried sigmoid(INPUT) * ( 1 - sigmoid(INPUT) ) which does not work. With RELU I tried to calculate its weights delta derivative with OUTPUT < 0 ? 0 : 1 but it does not work, no matter what. Here are my activation functions and rest of the code is on CODEPEN, I tryed to comment it well and make as easy as possible:

 activate (x) {
    if (this.aFunc == "relu") {
        //relu
        return x < 0 ? 0 : x;
    } else if (this.aFunc == "sigmoid") {
        //sigmoid  
        return 1 / (1 + Math.exp(-x));
    } else if (this.aFunc == "tanh") {
        //TanH
        return Math.tanh(x)
    } else {
        //binary step threshold
        return x < 0 ? 0 : 1;
    }
}