two important types of artificial neuron :the perceptron and the sigmoid neuron

Perceptrons

感知机的输入个数不限,每个输入的取值都是二元的(0或1,这点不确定，后续确认下)，输出是0或1.

Sigmoid neuron

Sigmoid neurons are similar to perceptrons, but modified so that small changes in their weights and bias cause only a small change in their output. That's the crucial fact which will allow a network of sigmoid neurons to learn.

sigmoid neuron 输入:these inputs can  take on any values between 0 and 1.

 

sigmoid neuron与perceptrons的相似点：当z = w . x + b ->正无穷时,sigmoid neuron的输出趋向于1.当z = w . x + b ->负无穷时,sigmoid neuron的输出趋向于0.而感知机的输出就是0或1.

The smoothness of σ (sigmoid函数值的平滑性) means that small changes  in the weights and in the bias will produce a small change in the output from the neuron.

上面公式如何推导出来？

后续的笔记直接写在纸上了

The architecture of neural networks

A simple network to classify handwritten digits

Learning with gradient descent

Implementing our network to classify digits

Toward deep learning

 

---