{ "cells": [ { "cell_type": "markdown", "id": "dc6818b8", "metadata": {}, "source": [ "# Texas RRC Inspection Expenses Analysis\n", "\n", "**Research question:** Does organizational capacity (budget, staffing) predict better regulatory outputs (inspections, compliance, enforcement), and how is that relationship moderated by goal ambiguity, district-level heterogeneity, and spatial/geographic factors?\n", "\n", "## Hypotheses\n", "- **H1 — Capacity → Outputs:** Higher OGI budget and FTE predict more inspections, higher compliance rates, and faster violation resolution.\n", "- **H2 — Goal Ambiguity:** When a larger share of RRC budget goes to the more ambiguous \"Energy Resource Development\" goal, the capacity → output relationship weakens.\n", "- **H3 — Multilevel / District Effects:** The capacity → output relationship varies across RRC districts (budget slope heterogeneity).\n", "- **H4 — Spatial & Geographic:** Offshore-jurisdiction and border districts moderate the capacity → output relationship; spatial autocorrelation in residuals is tested via Moran's I.\n", "\n", "**Data:**\n", "- PostgreSQL warehouse (`texas_data`): `inspections`, `violations`, `well_shape_tract`\n", "- `RRC Budget Data.xlsx`: statewide RRC budget by strategy, 2016–2024\n", "- Analysis period: 2016–2023 (2024 is budget estimate, excluded from regressions)\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "3ed415f0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: jupyter in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (1.1.1)\n", "Requirement already satisfied: ipykernel in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (6.31.0)\n", "Requirement already satisfied: pandas in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (2.3.3)\n", "Requirement already satisfied: numpy in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (2.0.2)\n", "Requirement already satisfied: sqlalchemy in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (2.0.47)\n", "Requirement already satisfied: psycopg2-binary in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (2.9.11)\n", "Requirement already satisfied: python-dotenv in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 7)) (1.2.1)\n", "Requirement already satisfied: openpyxl in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 8)) (3.1.5)\n", "Requirement already satisfied: statsmodels in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 9)) (0.14.6)\n", "Requirement already satisfied: scipy in ./.venv/lib/python3.9/site-packages (from -r requirements.txt (line 10)) (1.13.1)\n", "Collecting matplotlib\n", " Using cached matplotlib-3.9.4-cp39-cp39-macosx_11_0_arm64.whl (7.8 MB)\n", "Collecting seaborn\n", " Using cached seaborn-0.13.2-py3-none-any.whl (294 kB)\n", "Requirement already satisfied: notebook in ./.venv/lib/python3.9/site-packages (from jupyter->-r requirements.txt (line 1)) (7.5.4)\n", "Requirement already satisfied: jupyter-console in ./.venv/lib/python3.9/site-packages (from jupyter->-r requirements.txt (line 1)) (6.6.3)\n", "Requirement already satisfied: ipywidgets in ./.venv/lib/python3.9/site-packages (from jupyter->-r requirements.txt (line 1)) (8.1.8)\n", "Requirement already satisfied: nbconvert in ./.venv/lib/python3.9/site-packages (from jupyter->-r requirements.txt (line 1)) (7.17.0)\n", "Requirement already satisfied: jupyterlab in ./.venv/lib/python3.9/site-packages (from jupyter->-r requirements.txt (line 1)) (4.5.5)\n", "Requirement already satisfied: matplotlib-inline>=0.1 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (0.2.1)\n", "Requirement already satisfied: nest-asyncio>=1.4 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (1.6.0)\n", "Requirement already satisfied: jupyter-core!=5.0.*,>=4.12 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (5.8.1)\n", "Requirement already satisfied: packaging>=22 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (26.0)\n", "Requirement already satisfied: debugpy>=1.6.5 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (1.8.20)\n", "Requirement already satisfied: appnope>=0.1.2 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (0.1.4)\n", "Requirement already satisfied: pyzmq>=25 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (27.1.0)\n", "Requirement already satisfied: comm>=0.1.1 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (0.2.3)\n", "Requirement already satisfied: jupyter-client>=8.0.0 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (8.6.3)\n", "Requirement already satisfied: tornado>=6.2 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (6.5.4)\n", "Requirement already satisfied: traitlets>=5.4.0 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (5.14.3)\n", "Requirement already satisfied: ipython>=7.23.1 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (8.18.1)\n", "Requirement already satisfied: psutil>=5.7 in ./.venv/lib/python3.9/site-packages (from ipykernel->-r requirements.txt (line 2)) (7.2.2)\n", "Requirement already satisfied: tzdata>=2022.7 in ./.venv/lib/python3.9/site-packages (from pandas->-r requirements.txt (line 3)) (2025.3)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in ./.venv/lib/python3.9/site-packages (from pandas->-r requirements.txt (line 3)) (2.9.0.post0)\n", "Requirement already satisfied: pytz>=2020.1 in ./.venv/lib/python3.9/site-packages (from pandas->-r requirements.txt (line 3)) (2025.2)\n", "Requirement already satisfied: typing-extensions>=4.6.0 in ./.venv/lib/python3.9/site-packages (from sqlalchemy->-r requirements.txt (line 5)) (4.15.0)\n", "Requirement already satisfied: et-xmlfile in ./.venv/lib/python3.9/site-packages (from openpyxl->-r requirements.txt (line 8)) (2.0.0)\n", "Requirement already satisfied: patsy>=0.5.6 in ./.venv/lib/python3.9/site-packages (from statsmodels->-r requirements.txt (line 9)) (1.0.2)\n", "Collecting importlib-resources>=3.2.0\n", " Using cached importlib_resources-6.5.2-py3-none-any.whl (37 kB)\n", "Collecting kiwisolver>=1.3.1\n", " Using cached kiwisolver-1.4.7-cp39-cp39-macosx_11_0_arm64.whl (64 kB)\n", "Collecting pillow>=8\n", " Using cached pillow-11.3.0-cp39-cp39-macosx_11_0_arm64.whl (4.7 MB)\n", "Collecting pyparsing>=2.3.1\n", " Downloading pyparsing-3.3.2-py3-none-any.whl (122 kB)\n", "\u001b[K |████████████████████████████████| 122 kB 10.1 MB/s eta 0:00:01\n", "\u001b[?25hCollecting cycler>=0.10\n", " Using cached cycler-0.12.1-py3-none-any.whl (8.3 kB)\n", "Collecting contourpy>=1.0.1\n", " Using cached contourpy-1.3.0-cp39-cp39-macosx_11_0_arm64.whl (249 kB)\n", "Collecting fonttools>=4.22.0\n", " Using cached fonttools-4.60.2-cp39-cp39-macosx_10_9_universal2.whl (2.9 MB)\n", "Requirement already satisfied: zipp>=3.1.0 in ./.venv/lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib->-r requirements.txt (line 11)) (3.23.0)\n", "Requirement already satisfied: prompt-toolkit<3.1.0,>=3.0.41 in ./.venv/lib/python3.9/site-packages (from ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (3.0.52)\n", "Requirement already satisfied: decorator in ./.venv/lib/python3.9/site-packages (from ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (5.2.1)\n", "Requirement already satisfied: pexpect>4.3 in ./.venv/lib/python3.9/site-packages (from ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (4.9.0)\n", "Requirement already satisfied: pygments>=2.4.0 in ./.venv/lib/python3.9/site-packages (from ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (2.19.2)\n", "Requirement already satisfied: jedi>=0.16 in ./.venv/lib/python3.9/site-packages (from ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (0.19.2)\n", "Requirement already satisfied: exceptiongroup in ./.venv/lib/python3.9/site-packages (from ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (1.3.1)\n", "Requirement already satisfied: stack-data in ./.venv/lib/python3.9/site-packages (from ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (0.6.3)\n", "Requirement already satisfied: parso<0.9.0,>=0.8.4 in ./.venv/lib/python3.9/site-packages (from jedi>=0.16->ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (0.8.6)\n", "Requirement already satisfied: importlib-metadata>=4.8.3 in ./.venv/lib/python3.9/site-packages (from jupyter-client>=8.0.0->ipykernel->-r requirements.txt (line 2)) (8.7.1)\n", "Requirement already satisfied: platformdirs>=2.5 in ./.venv/lib/python3.9/site-packages (from jupyter-core!=5.0.*,>=4.12->ipykernel->-r requirements.txt (line 2)) (4.4.0)\n", "Requirement already satisfied: ptyprocess>=0.5 in ./.venv/lib/python3.9/site-packages (from pexpect>4.3->ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (0.7.0)\n", "Requirement already satisfied: wcwidth in ./.venv/lib/python3.9/site-packages (from prompt-toolkit<3.1.0,>=3.0.41->ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (0.6.0)\n", "Requirement already satisfied: six>=1.5 in ./.venv/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->-r requirements.txt (line 3)) (1.17.0)\n", "Requirement already satisfied: jupyterlab_widgets~=3.0.15 in ./.venv/lib/python3.9/site-packages (from ipywidgets->jupyter->-r requirements.txt (line 1)) (3.0.16)\n", "Requirement already satisfied: widgetsnbextension~=4.0.14 in ./.venv/lib/python3.9/site-packages (from ipywidgets->jupyter->-r requirements.txt (line 1)) (4.0.15)\n", "Requirement already satisfied: jupyter-lsp>=2.0.0 in ./.venv/lib/python3.9/site-packages (from jupyterlab->jupyter->-r requirements.txt (line 1)) (2.3.0)\n", "Requirement already satisfied: notebook-shim>=0.2 in ./.venv/lib/python3.9/site-packages (from jupyterlab->jupyter->-r requirements.txt (line 1)) (0.2.4)\n", "Requirement already satisfied: jupyterlab-server<3,>=2.28.0 in ./.venv/lib/python3.9/site-packages (from jupyterlab->jupyter->-r requirements.txt (line 1)) (2.28.0)\n", "Requirement already satisfied: httpx<1,>=0.25.0 in ./.venv/lib/python3.9/site-packages (from jupyterlab->jupyter->-r requirements.txt (line 1)) (0.28.1)\n", "Requirement already satisfied: jinja2>=3.0.3 in ./.venv/lib/python3.9/site-packages (from jupyterlab->jupyter->-r requirements.txt (line 1)) (3.1.6)\n", "Requirement already satisfied: tomli>=1.2.2 in ./.venv/lib/python3.9/site-packages (from jupyterlab->jupyter->-r requirements.txt (line 1)) (2.4.0)\n", "Requirement already satisfied: jupyter-server<3,>=2.4.0 in ./.venv/lib/python3.9/site-packages (from jupyterlab->jupyter->-r requirements.txt (line 1)) (2.17.0)\n", "Requirement already satisfied: async-lru>=1.0.0 in ./.venv/lib/python3.9/site-packages (from jupyterlab->jupyter->-r requirements.txt (line 1)) (2.0.5)\n", "Requirement already satisfied: setuptools>=41.1.0 in ./.venv/lib/python3.9/site-packages (from jupyterlab->jupyter->-r requirements.txt (line 1)) (58.0.4)\n", "Requirement already satisfied: certifi in ./.venv/lib/python3.9/site-packages (from httpx<1,>=0.25.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (2026.2.25)\n", "Requirement already satisfied: httpcore==1.* in ./.venv/lib/python3.9/site-packages (from httpx<1,>=0.25.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (1.0.9)\n", "Requirement already satisfied: anyio in ./.venv/lib/python3.9/site-packages (from httpx<1,>=0.25.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (4.12.1)\n", "Requirement already satisfied: idna in ./.venv/lib/python3.9/site-packages (from httpx<1,>=0.25.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (3.11)\n", "Requirement already satisfied: h11>=0.16 in ./.venv/lib/python3.9/site-packages (from httpcore==1.*->httpx<1,>=0.25.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.16.0)\n", "Requirement already satisfied: MarkupSafe>=2.0 in ./.venv/lib/python3.9/site-packages (from jinja2>=3.0.3->jupyterlab->jupyter->-r requirements.txt (line 1)) (3.0.3)\n", "Requirement already satisfied: jupyter-server-terminals>=0.4.4 in ./.venv/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.5.4)\n", "Requirement already satisfied: nbformat>=5.3.0 in ./.venv/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (5.10.4)\n", "Requirement already satisfied: send2trash>=1.8.2 in ./.venv/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (2.1.0)\n", "Requirement already satisfied: argon2-cffi>=21.1 in ./.venv/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (25.1.0)\n", "Requirement already satisfied: websocket-client>=1.7 in ./.venv/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (1.9.0)\n", "Requirement already satisfied: jupyter-events>=0.11.0 in ./.venv/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.12.0)\n", "Requirement already satisfied: overrides>=5.0 in ./.venv/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (7.7.0)\n", "Requirement already satisfied: terminado>=0.8.3 in ./.venv/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.18.1)\n", "Requirement already satisfied: prometheus-client>=0.9 in ./.venv/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.24.1)\n", "Requirement already satisfied: argon2-cffi-bindings in ./.venv/lib/python3.9/site-packages (from argon2-cffi>=21.1->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (25.1.0)\n", "Requirement already satisfied: pyyaml>=5.3 in ./.venv/lib/python3.9/site-packages (from jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (6.0.3)\n", "Requirement already satisfied: python-json-logger>=2.0.4 in ./.venv/lib/python3.9/site-packages (from jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (4.0.0)\n", "Requirement already satisfied: referencing in ./.venv/lib/python3.9/site-packages (from jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.36.2)\n", "Requirement already satisfied: jsonschema[format-nongpl]>=4.18.0 in ./.venv/lib/python3.9/site-packages (from jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (4.25.1)\n", "Requirement already satisfied: rfc3986-validator>=0.1.1 in ./.venv/lib/python3.9/site-packages (from jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.1.1)\n", "Requirement already satisfied: rfc3339-validator in ./.venv/lib/python3.9/site-packages (from jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.1.4)\n", "Requirement already satisfied: attrs>=22.2.0 in ./.venv/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (25.4.0)\n", "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in ./.venv/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (2025.9.1)\n", "Requirement already satisfied: rpds-py>=0.7.1 in ./.venv/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.27.1)\n", "Requirement already satisfied: webcolors>=24.6.0 in ./.venv/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (24.11.1)\n", "Requirement already satisfied: isoduration in ./.venv/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (20.11.0)\n", "Requirement already satisfied: fqdn in ./.venv/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (1.5.1)\n", "Requirement already satisfied: jsonpointer>1.13 in ./.venv/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (3.0.0)\n", "Requirement already satisfied: uri-template in ./.venv/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (1.3.0)\n", "Requirement already satisfied: rfc3987-syntax>=1.1.0 in ./.venv/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (1.1.0)\n", "Requirement already satisfied: babel>=2.10 in ./.venv/lib/python3.9/site-packages (from jupyterlab-server<3,>=2.28.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (2.18.0)\n", "Requirement already satisfied: json5>=0.9.0 in ./.venv/lib/python3.9/site-packages (from jupyterlab-server<3,>=2.28.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (0.13.0)\n", "Requirement already satisfied: requests>=2.31 in ./.venv/lib/python3.9/site-packages (from jupyterlab-server<3,>=2.28.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (2.32.5)\n", "Requirement already satisfied: bleach[css]!=5.0.0 in ./.venv/lib/python3.9/site-packages (from nbconvert->jupyter->-r requirements.txt (line 1)) (6.2.0)\n", "Requirement already satisfied: defusedxml in ./.venv/lib/python3.9/site-packages (from nbconvert->jupyter->-r requirements.txt (line 1)) (0.7.1)\n", "Requirement already satisfied: jupyterlab-pygments in ./.venv/lib/python3.9/site-packages (from nbconvert->jupyter->-r requirements.txt (line 1)) (0.3.0)\n", "Requirement already satisfied: nbclient>=0.5.0 in ./.venv/lib/python3.9/site-packages (from nbconvert->jupyter->-r requirements.txt (line 1)) (0.10.2)\n", "Requirement already satisfied: beautifulsoup4 in ./.venv/lib/python3.9/site-packages (from nbconvert->jupyter->-r requirements.txt (line 1)) (4.14.3)\n", "Requirement already satisfied: pandocfilters>=1.4.1 in ./.venv/lib/python3.9/site-packages (from nbconvert->jupyter->-r requirements.txt (line 1)) (1.5.1)\n", "Requirement already satisfied: mistune<4,>=2.0.3 in ./.venv/lib/python3.9/site-packages (from nbconvert->jupyter->-r requirements.txt (line 1)) (3.2.0)\n", "Requirement already satisfied: webencodings in ./.venv/lib/python3.9/site-packages (from bleach[css]!=5.0.0->nbconvert->jupyter->-r requirements.txt (line 1)) (0.5.1)\n", "Requirement already satisfied: tinycss2<1.5,>=1.1.0 in ./.venv/lib/python3.9/site-packages (from bleach[css]!=5.0.0->nbconvert->jupyter->-r requirements.txt (line 1)) (1.4.0)\n", "Requirement already satisfied: fastjsonschema>=2.15 in ./.venv/lib/python3.9/site-packages (from nbformat>=5.3.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (2.21.2)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in ./.venv/lib/python3.9/site-packages (from requests>=2.31->jupyterlab-server<3,>=2.28.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (2.6.3)\n", "Requirement already satisfied: charset_normalizer<4,>=2 in ./.venv/lib/python3.9/site-packages (from requests>=2.31->jupyterlab-server<3,>=2.28.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (3.4.4)\n", "Requirement already satisfied: lark>=1.2.2 in ./.venv/lib/python3.9/site-packages (from rfc3987-syntax>=1.1.0->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (1.3.1)\n", "Requirement already satisfied: cffi>=1.0.1 in ./.venv/lib/python3.9/site-packages (from argon2-cffi-bindings->argon2-cffi>=21.1->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (2.0.0)\n", "Requirement already satisfied: pycparser in ./.venv/lib/python3.9/site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi>=21.1->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (2.23)\n", "Requirement already satisfied: soupsieve>=1.6.1 in ./.venv/lib/python3.9/site-packages (from beautifulsoup4->nbconvert->jupyter->-r requirements.txt (line 1)) (2.8.3)\n", "Requirement already satisfied: arrow>=0.15.0 in ./.venv/lib/python3.9/site-packages (from isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.11.0->jupyter-server<3,>=2.4.0->jupyterlab->jupyter->-r requirements.txt (line 1)) (1.4.0)\n", "Requirement already satisfied: asttokens>=2.1.0 in ./.venv/lib/python3.9/site-packages (from stack-data->ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (3.0.1)\n", "Requirement already satisfied: executing>=1.2.0 in ./.venv/lib/python3.9/site-packages (from stack-data->ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (2.2.1)\n", "Requirement already satisfied: pure-eval in ./.venv/lib/python3.9/site-packages (from stack-data->ipython>=7.23.1->ipykernel->-r requirements.txt (line 2)) (0.2.3)\n", "Installing collected packages: pyparsing, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib, seaborn\n", "Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 matplotlib-3.9.4 pillow-11.3.0 pyparsing-3.3.2 seaborn-0.13.2\n", "\u001b[33mWARNING: You are using pip version 21.2.4; however, version 26.0.1 is available.\n", "You should consider upgrading via the '/Users/dpadams/Repos/texas-inspection-expenses/.venv/bin/python -m pip install --upgrade pip' command.\u001b[0m\n", "Note: you may need to restart the kernel to use updated packages.\n" ] } ], "source": [ "%pip install -r requirements.txt" ] }, { "cell_type": "code", "execution_count": 5, "id": "49de2b5c", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Matplotlib is building the font cache; this may take a moment.\n" ] } ], "source": [ "import os\n", "import warnings\n", "from pathlib import Path\n", "from urllib.parse import quote_plus\n", "\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import matplotlib.ticker as mticker\n", "import statsmodels.formula.api as smf\n", "from dotenv import load_dotenv\n", "from sqlalchemy import create_engine, text\n", "from scipy.spatial.distance import cdist\n", "\n", "warnings.filterwarnings(\"ignore\", category=UserWarning)\n", "pd.set_option(\"display.float_format\", \"{:,.2f}\".format)\n" ] }, { "cell_type": "code", "execution_count": 9, "id": "08420da3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Connected → texas_data on localhost:5433\n" ] } ], "source": [ "load_dotenv(override=False)\n", "\n", "host = os.getenv(\"PGHOST\", \"localhost\")\n", "port = os.getenv(\"5433\", \"5433\")\n", "user = os.getenv(\"PGUSER\", \"postgres\")\n", "password = quote_plus(os.getenv(\"PGPASSWORD\", \"\"))\n", "database = os.getenv(\"PGDATABASE\", \"texas_data\")\n", "\n", "engine = create_engine(\n", " f\"postgresql+psycopg2://{user}:{password}@{host}:{port}/{database}\"\n", ")\n", "print(f\"Connected → {database} on {host}:{port}\")\n" ] }, { "cell_type": "markdown", "id": "b4e6c44f", "metadata": {}, "source": [ "## 1. Data Loading" ] }, { "cell_type": "code", "execution_count": null, "id": "43886f13", "metadata": {}, "outputs": [], "source": [ "# District-year inspection metrics aggregated in SQL.\n", "# LAG() computes days since the previous inspection for the same well (api_norm).\n", "insp_sql = \"\"\"\n", "WITH lagged AS (\n", " SELECT\n", " district,\n", " EXTRACT(year FROM inspection_date)::int AS year,\n", " api_norm,\n", " inspection_date,\n", " CASE WHEN UPPER(compliance::text) IN ('YES', 'Y') THEN 1.0 ELSE 0.0 END AS is_compliant,\n", " EXTRACT(EPOCH FROM (\n", " inspection_date\n", " - LAG(inspection_date) OVER (PARTITION BY api_norm ORDER BY inspection_date)\n", " )) / 86400.0 AS days_since_prev\n", " FROM inspections\n", " WHERE inspection_date IS NOT NULL\n", " AND district IS NOT NULL\n", " AND EXTRACT(year FROM inspection_date) BETWEEN 2016 AND 2025\n", ")\n", "SELECT\n", " district,\n", " year,\n", " COUNT(*) AS total_inspections,\n", " COUNT(DISTINCT api_norm) AS unique_wells,\n", " ROUND(AVG(is_compliant)::numeric * 100, 2) AS compliance_rate,\n", " ROUND(AVG(days_since_prev)::numeric, 1) AS avg_days_between_inspections\n", "FROM lagged\n", "GROUP BY district, year\n", "ORDER BY district, year\n", "\"\"\"\n", "\n", "insp = pd.read_sql(text(insp_sql), engine)\n", "print(f\"Inspections panel: {len(insp):,} district-year rows | {insp['district'].nunique()} districts\")\n", "insp.head()\n" ] }, { "cell_type": "code", "execution_count": null, "id": "3841e2f5", "metadata": {}, "outputs": [], "source": [ "# District-year violation metrics. Blank last_enf_action strings treated as no action.\n", "viol_sql = \"\"\"\n", "SELECT\n", " district,\n", " EXTRACT(year FROM violation_disc_date)::int AS year,\n", " COUNT(*) AS total_violations,\n", " COUNT(DISTINCT api_norm) AS unique_wells_with_violations,\n", " SUM(CASE WHEN major_viol_ind = 'Y' THEN 1 ELSE 0 END) AS major_violations,\n", " ROUND(AVG(CASE WHEN compliant_on_reinsp = 'Y' THEN 1.0 ELSE 0.0 END)::numeric * 100, 2)\n", " AS resolution_rate,\n", " ROUND(AVG(CASE WHEN last_enf_action IS NOT NULL AND last_enf_action <> ''\n", " THEN 1.0 ELSE 0.0 END)::numeric * 100, 2) AS enforcement_rate,\n", " ROUND(AVG(\n", " CASE WHEN last_enf_action_date IS NOT NULL\n", " THEN EXTRACT(EPOCH FROM (last_enf_action_date - violation_disc_date)) / 86400.0\n", " END\n", " )::numeric, 1) AS avg_days_to_enforcement\n", "FROM violations\n", "WHERE violation_disc_date IS NOT NULL\n", " AND district IS NOT NULL\n", " AND EXTRACT(year FROM violation_disc_date) BETWEEN 2016 AND 2025\n", "GROUP BY district, year\n", "ORDER BY district, year\n", "\"\"\"\n", "\n", "viol = pd.read_sql(text(viol_sql), engine)\n", "print(f\"Violations panel: {len(viol):,} district-year rows\")\n", "viol.head()\n" ] }, { "cell_type": "code", "execution_count": 12, "id": "9e196cac", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Budget long: 18 rows (2 strategies × 9 years)\n" ] }, { "data": { "application/vnd.microsoft.datawrangler.viewer.v0+json": { "columns": [ { "name": "index", "rawType": "int64", "type": "integer" }, { "name": "year", "rawType": "int64", "type": "integer" }, { "name": "strategy", "rawType": "object", "type": "string" }, { "name": "total_budget", "rawType": "float64", "type": "float" }, { "name": "salaries", "rawType": "float64", "type": "float" }, { "name": "other_personnel", "rawType": "float64", "type": "float" }, { "name": "professional_fees", "rawType": "float64", "type": "float" }, { "name": "travel", "rawType": "float64", "type": "float" }, { "name": "other_operating", "rawType": "float64", "type": "float" }, { "name": "capital_exp", "rawType": "float64", "type": "float" }, { "name": "fte", "rawType": "float64", "type": "float" } ], "ref": "d1e3d3b9-d373-4e93-9e56-96a3f8488e43", "rows": [ [ "0", "2016", "Energy Resource Development", "11708475.0", "7669719.0", "398589.0", "3366389.0", "16477.0", "210293.0", "0.0", "130.6" ], [ "1", "2017", "Energy Resource Development", "10911094.0", "7273775.0", "389348.0", "3118066.0", "6792.0", "77855.0", "0.0", "120.3" ], [ "2", "2018", "Energy Resource Development", "9846886.0", "7292933.0", "282337.0", "977645.0", "28694.0", "1045727.0", "0.0", "131.0" ], [ "3", "2019", "Energy Resource Development", "11123757.0", "8068497.0", "217988.0", "1493755.0", "73651.0", "988740.0", "13232.0", "137.4" ], [ "4", "2020", "Energy Resource Development", "17280569.0", "9707894.0", "236356.0", "5989236.0", "41752.0", "1165481.0", "54037.0", "153.4" ], [ "5", "2021", "Energy Resource Development", "16237704.0", "10887561.0", "237777.0", "3562816.0", "5614.0", "1446301.0", "10140.0", "168.1" ], [ "6", "2022", "Energy Resource Development", "25583205.0", "11166309.0", "246340.0", "12560550.0", "37731.0", "1246443.0", "19985.0", "157.1" ], [ "7", "2023", "Energy Resource Development", "26903564.0", "11056060.0", "252933.0", "12846821.0", "56650.0", "2287481.0", "48344.0", "151.3" ], [ "8", "2024", "Energy Resource Development", "35533565.0", "13183578.0", "229161.0", "15140585.0", "144641.0", "6425653.0", "0.0", "186.0" ], [ "9", "2016", "Oil/Gas Monitoring & Inspections", "18471666.0", "15080122.0", "685768.0", "1546321.0", "22630.0", "208311.0", "121363.0", "256.7" ], [ "10", "2017", "Oil/Gas Monitoring & Inspections", "17204058.0", "15086262.0", "686194.0", "176786.0", "19654.0", "230525.0", "272461.0", "249.5" ], [ "11", "2018", "Oil/Gas Monitoring & Inspections", "17562431.0", "13083406.0", "430429.0", "1147080.0", "57312.0", "1040639.0", "649172.0", "229.9" ], [ "12", "2019", "Oil/Gas Monitoring & Inspections", "21951747.0", "14878875.0", "340135.0", "2895436.0", "187048.0", "1185772.0", "1255930.0", "255.6" ], [ "13", "2020", "Oil/Gas Monitoring & Inspections", "26057560.0", "17228302.0", "417683.0", "4822351.0", "106428.0", "1398705.0", "896846.0", "284.0" ], [ "14", "2021", "Oil/Gas Monitoring & Inspections", "28756689.0", "17155864.0", "426139.0", "8212873.0", "34762.0", "1394783.0", "230439.0", "277.8" ], [ "15", "2022", "Oil/Gas Monitoring & Inspections", "25914265.0", "17834460.0", "391138.0", "4007178.0", "154334.0", "1255945.0", "694706.0", "264.0" ], [ "16", "2023", "Oil/Gas Monitoring & Inspections", "34330858.0", "18622389.0", "457753.0", "8945350.0", "149418.0", "2428330.0", "2234623.0", "271.2" ], [ "17", "2024", "Oil/Gas Monitoring & Inspections", "38506556.0", "20834721.0", "361687.0", "8851915.0", "316806.0", "4112998.0", "2659208.0", "280.8" ] ], "shape": { "columns": 10, "rows": 18 } }, "text/html": [ "
| \n", " | year | \n", "strategy | \n", "total_budget | \n", "salaries | \n", "other_personnel | \n", "professional_fees | \n", "travel | \n", "other_operating | \n", "capital_exp | \n", "fte | \n", "
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "2016 | \n", "Energy Resource Development | \n", "11,708,475.00 | \n", "7,669,719.00 | \n", "398,589.00 | \n", "3,366,389.00 | \n", "16,477.00 | \n", "210,293.00 | \n", "0.00 | \n", "130.60 | \n", "
| 1 | \n", "2017 | \n", "Energy Resource Development | \n", "10,911,094.00 | \n", "7,273,775.00 | \n", "389,348.00 | \n", "3,118,066.00 | \n", "6,792.00 | \n", "77,855.00 | \n", "0.00 | \n", "120.30 | \n", "
| 2 | \n", "2018 | \n", "Energy Resource Development | \n", "9,846,886.00 | \n", "7,292,933.00 | \n", "282,337.00 | \n", "977,645.00 | \n", "28,694.00 | \n", "1,045,727.00 | \n", "0.00 | \n", "131.00 | \n", "
| 3 | \n", "2019 | \n", "Energy Resource Development | \n", "11,123,757.00 | \n", "8,068,497.00 | \n", "217,988.00 | \n", "1,493,755.00 | \n", "73,651.00 | \n", "988,740.00 | \n", "13,232.00 | \n", "137.40 | \n", "
| 4 | \n", "2020 | \n", "Energy Resource Development | \n", "17,280,569.00 | \n", "9,707,894.00 | \n", "236,356.00 | \n", "5,989,236.00 | \n", "41,752.00 | \n", "1,165,481.00 | \n", "54,037.00 | \n", "153.40 | \n", "
| 5 | \n", "2021 | \n", "Energy Resource Development | \n", "16,237,704.00 | \n", "10,887,561.00 | \n", "237,777.00 | \n", "3,562,816.00 | \n", "5,614.00 | \n", "1,446,301.00 | \n", "10,140.00 | \n", "168.10 | \n", "
| 6 | \n", "2022 | \n", "Energy Resource Development | \n", "25,583,205.00 | \n", "11,166,309.00 | \n", "246,340.00 | \n", "12,560,550.00 | \n", "37,731.00 | \n", "1,246,443.00 | \n", "19,985.00 | \n", "157.10 | \n", "
| 7 | \n", "2023 | \n", "Energy Resource Development | \n", "26,903,564.00 | \n", "11,056,060.00 | \n", "252,933.00 | \n", "12,846,821.00 | \n", "56,650.00 | \n", "2,287,481.00 | \n", "48,344.00 | \n", "151.30 | \n", "
| 8 | \n", "2024 | \n", "Energy Resource Development | \n", "35,533,565.00 | \n", "13,183,578.00 | \n", "229,161.00 | \n", "15,140,585.00 | \n", "144,641.00 | \n", "6,425,653.00 | \n", "0.00 | \n", "186.00 | \n", "
| 9 | \n", "2016 | \n", "Oil/Gas Monitoring & Inspections | \n", "18,471,666.00 | \n", "15,080,122.00 | \n", "685,768.00 | \n", "1,546,321.00 | \n", "22,630.00 | \n", "208,311.00 | \n", "121,363.00 | \n", "256.70 | \n", "
| 10 | \n", "2017 | \n", "Oil/Gas Monitoring & Inspections | \n", "17,204,058.00 | \n", "15,086,262.00 | \n", "686,194.00 | \n", "176,786.00 | \n", "19,654.00 | \n", "230,525.00 | \n", "272,461.00 | \n", "249.50 | \n", "
| 11 | \n", "2018 | \n", "Oil/Gas Monitoring & Inspections | \n", "17,562,431.00 | \n", "13,083,406.00 | \n", "430,429.00 | \n", "1,147,080.00 | \n", "57,312.00 | \n", "1,040,639.00 | \n", "649,172.00 | \n", "229.90 | \n", "
| 12 | \n", "2019 | \n", "Oil/Gas Monitoring & Inspections | \n", "21,951,747.00 | \n", "14,878,875.00 | \n", "340,135.00 | \n", "2,895,436.00 | \n", "187,048.00 | \n", "1,185,772.00 | \n", "1,255,930.00 | \n", "255.60 | \n", "
| 13 | \n", "2020 | \n", "Oil/Gas Monitoring & Inspections | \n", "26,057,560.00 | \n", "17,228,302.00 | \n", "417,683.00 | \n", "4,822,351.00 | \n", "106,428.00 | \n", "1,398,705.00 | \n", "896,846.00 | \n", "284.00 | \n", "
| 14 | \n", "2021 | \n", "Oil/Gas Monitoring & Inspections | \n", "28,756,689.00 | \n", "17,155,864.00 | \n", "426,139.00 | \n", "8,212,873.00 | \n", "34,762.00 | \n", "1,394,783.00 | \n", "230,439.00 | \n", "277.80 | \n", "
| 15 | \n", "2022 | \n", "Oil/Gas Monitoring & Inspections | \n", "25,914,265.00 | \n", "17,834,460.00 | \n", "391,138.00 | \n", "4,007,178.00 | \n", "154,334.00 | \n", "1,255,945.00 | \n", "694,706.00 | \n", "264.00 | \n", "
| 16 | \n", "2023 | \n", "Oil/Gas Monitoring & Inspections | \n", "34,330,858.00 | \n", "18,622,389.00 | \n", "457,753.00 | \n", "8,945,350.00 | \n", "149,418.00 | \n", "2,428,330.00 | \n", "2,234,623.00 | \n", "271.20 | \n", "
| 17 | \n", "2024 | \n", "Oil/Gas Monitoring & Inspections | \n", "38,506,556.00 | \n", "20,834,721.00 | \n", "361,687.00 | \n", "8,851,915.00 | \n", "316,806.00 | \n", "4,112,998.00 | \n", "2,659,208.00 | \n", "280.80 | \n", "