{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Notebook 4: Deep Learning, Artificial Neurons & Language Models (LLMs)\n",
        "\n",
        "In Chapter 5, we explored **Neural Networks and Deep Learning**, learning how simple biological inspiration led to artificial perceptron nodes, backpropagation, and state-of-the-art Generative Language Models (LLMs).\n",
        "\n",
        "In this final hands-on notebook, we simulate a working Artificial Neuron and build a miniature text prediction LLM!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Experiment 1: Simulating an Artificial Neuron (Perceptron)\n",
        "\n",
        "An artificial node (or neuron) takes inputs (`x1`, `x2`), multiplies each by a weight (`w1`, `w2`), adds a bias (`b`), and fires (activates) if the total reaches a threshold.\n",
        "\n",
        "Let us build a simple Neuron that decides whether to turn on an irrigation pump based on soil dryness (`input 1`) and crop need (`input 2`)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "class ArtificialNeuron:\n",
        "    def __init__(self, weight1, weight2, bias):\n",
        "        self.w1 = weight1\n",
        "        self.w2 = weight2\n",
        "        self.bias = bias\n",
        "        \n",
        "    def activate(self, x1, x2):\n",
        "        # Weighted sum plus bias\n",
        "        total = (x1 * self.w1) + (x2 * self.w2) + self.bias\n",
        "        # Activation threshold: if total > 0, output 1 (Turn Pump ON), else 0 (Keep OFF)\n",
        "        return 1 if total > 0 else 0, total\n",
        "\n",
        "# Let us initialize our neuron with learned weights\n",
        "pump_neuron = ArtificialNeuron(weight1=0.6, weight2=0.8, bias=-0.7)\n",
        "\n",
        "# Test with high dryness (0.9) and high crop need (0.8)\n",
        "decision, score = pump_neuron.activate(0.9, 0.8)\n",
        "print(f\"Neuron score: {score:.2f} -> Pump Decision: {'ON (1)' if decision else 'OFF (0)'}\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Experiment 2: Miniature Language Model (Predicting Next Words like ChatGPT)\n",
        "\n",
        "A Large Language Model (LLM) like Gemini or ChatGPT works on a core principle: given a sequence of words, predict what word comes next based on patterns learned from vast text data.\n",
        "\n",
        "Let us train a miniature language model on a short text and watch it generate sentences automatically!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# Our training corpus (Change these sentences and run again!)\n",
        "training_text = \"\"\"\n",
        "the village farmer goes to the market\n",
        "the village farmer grows grain in the field\n",
        "the children go to the school\n",
        "the children learn science at the school\n",
        "the market opens in the morning\n",
        "the school opens in the morning\n",
        "\"\"\"\n",
        "\n",
        "words = training_text.split()\n",
        "# Count which word follows which word\n",
        "follow_counts = {}\n",
        "for i in range(len(words) - 1):\n",
        "    curr, next_w = words[i], words[i + 1]\n",
        "    if curr not in follow_counts:\n",
        "        follow_counts[curr] = {}\n",
        "    follow_counts[curr][next_w] = follow_counts[curr].get(next_w, 0) + 1\n",
        "\n",
        "def predict_next_word(word):\n",
        "    counts = follow_counts.get(word)\n",
        "    if not counts:\n",
        "        return \"(unknown)\"\n",
        "    return max(counts, key=counts.get)\n",
        "\n",
        "# Let us generate a sentence starting from the word 'the'\n",
        "current_word = \"the\"\n",
        "generated = [current_word]\n",
        "for _ in range(7):\n",
        "    current_word = predict_next_word(current_word)\n",
        "    generated.append(current_word)\n",
        "\n",
        "print(\"Miniature LLM generated sentence:\", \" \".join(generated))"
      ],
      "execution_count": null,
      "outputs": []
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}