{"authors":[{"name":"Remco Loup"}],"description":"How Remco Loup, CEO/CTO of 333travel, looks at problems and solves them: with data, with teams, and with whatever technology fits.","favicon":"https://www.allsubjectsmatter.nl/images/icon.png","feed_url":"https://www.allsubjectsmatter.nl/feed.json","home_page_url":"https://www.allsubjectsmatter.nl/","icon":"https://www.allsubjectsmatter.nl/images/icon.png","items":[{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eIn \u003ca href=\"https://www.allsubjectsmatter.nl/can-a-conversation-be-a-travel-channel-we-built-the-architecture-to-find-out/\" data-preview\u003ePart 1\u003c/a\u003e, I wrote about why we built an AI travel interface from scratch, what we learned from our first failed attempt in 2019, and why a plugin can never get you to a fully integrated conversational experience. This part is the how. The architecture, the search pipeline, the memory problem that took four attempts to solve, and the UI/UX work that makes 25 seconds feel like 5.\u003c/p\u003e\n\u003ch2 id=\"the-architecture\"\u003eThe architecture\u003c/h2\u003e\n\u003cp\u003eSince this is a tech blog and not a marketing page, let me show you what\u0026rsquo;s actually running. This isn\u0026rsquo;t a weekend project. This is production infrastructure.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-text\"\u003e\u003ccode class=\"language-text\"\u003eFrontend (React \u0026#43; TypeScript \u0026#43; Zustand)\n    │\n    │  WebSocket (real-time, bidirectional)\n    │\nFastAPI Backend\n    ├── Query Orchestrator\n    │     ├── LLM Client (multi-provider with automatic fallback)\n    │     ├── Tool Registry (9 specialized tools)\n    │     └── Observer System (async post-turn analysis)\n    ├── RAG Pipeline\n    │     ├── Vector Database (3 content collections)\n    │     ├── Embedding Engine\n    │     └── Reranker (dual provider, switchable)\n    ├── Conversation Memory (dual storage)\n    │     ├── SQL (message history)\n    │     └── Vector DB (semantic search over past messages)\n    ├── Security Layer\n    │     ├── Input Guard (prompt injection defense)\n    │     ├── Output Guard (credential leak prevention)\n    │     └── Rate Limiting \u0026#43; IP Management\n    ├── MCP Server (Model Context Protocol)\n    │     └── External AI assistant access to our data\n    └── Lead Management\n          ├── Proposal Generation\n          └── Specialist Handoff\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eThe frontend is a React application with real-time WebSocket communication. No polling. No request-response cycles where you wait for a full answer. Every token streams in real-time, buffered at 50ms intervals to prevent UI flicker. Product results push to the interface the moment they\u0026rsquo;re found, before the LLM even starts composing its response.\u003c/p\u003e\n\u003cp\u003eThe backend is Python/FastAPI with a modular tool system. The LLM doesn\u0026rsquo;t just generate text. It has access to 9 specialized tools and decides autonomously which ones to use. Need to search trips? It calls \u003ccode\u003esearch_products\u003c/code\u003e. Need destination information? It calls \u003ccode\u003esearch_knowledge\u003c/code\u003e. Need to recall what was discussed earlier? It queries conversation memory semantically.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe LLM isn\u0026rsquo;t generating answers. It\u0026rsquo;s orchestrating tools and composing a response from real data.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe LLM layer runs with a multi-provider fallback system. If the primary provider goes down, the system automatically switches to the secondary. No downtime. No errors. The customer never notices.\u003c/p\u003e\n\u003cp\u003eAnd there\u0026rsquo;s already an MCP server running, the protocol I described in my Web 4.0 post. External AI assistants can already query our travel data through it. It\u0026rsquo;s early days, but the infrastructure for AI-to-business communication is live. When Lisa\u0026rsquo;s AI assistant wants to search 333travel\u0026rsquo;s products, the endpoint already exists.\u003c/p\u003e\n\u003ch3 id=\"the-rag-pipeline-where-the-real-work-happens\"\u003eThe RAG pipeline: where the real work happens\u003c/h3\u003e\n\u003cp\u003eRAG, Retrieval-Augmented Generation, is the backbone of the entire system. It\u0026rsquo;s what separates \u0026ldquo;GPT with a knowledge base\u0026rdquo; from \u0026ldquo;an AI that actually knows your product.\u0026rdquo; But RAG done poorly is almost worse than no RAG at all.\u003c/p\u003e\n\u003cp\u003eWe maintain three separate vector collections:\u003c/p\u003e\n\u003ctable\u003e\n\t\u003cthead\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003cth\u003eCollection\u003c/th\u003e\n\t\t\t\t\t\u003cth\u003eWhat\u0026rsquo;s in it\u003c/th\u003e\n\t\t\t\t\t\u003cth\u003eWhat it does\u003c/th\u003e\n\t\t\t\u003c/tr\u003e\n\t\u003c/thead\u003e\n\t\u003ctbody\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003ctd\u003eTravel Products\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003eRoundtrips, hotels, tours, cruises\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003ePowers the main product search\u003c/td\u003e\n\t\t\t\u003c/tr\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003ctd\u003eTravel Details\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003eDay programs, excursions, inclusions\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003eAnswers specific itinerary questions\u003c/td\u003e\n\t\t\t\u003c/tr\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003ctd\u003eTravel Knowledge\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003eBlogs, destination guides, travel info\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003eProvides context and inspiration\u003c/td\u003e\n\t\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eWhen a customer asks something, the query gets embedded and thrown against the relevant collection. But here\u0026rsquo;s the critical part: \u003cstrong\u003ewe don\u0026rsquo;t just take the top vector matches and call it a day.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eVector similarity alone is mediocre. It gets you in the right ballpark, but it doesn\u0026rsquo;t get you the best results. So we do a broad fetch, significantly more results than we need, and then run them through a reranker. The reranker evaluates each query-document pair for actual semantic relevance, not just embedding proximity.\u003c/p\u003e\n\u003cp\u003eThe difference is enormous. It\u0026rsquo;s the difference between \u0026ldquo;here are trips that contain some of the words you used\u0026rdquo; and \u0026ldquo;here are trips that actually match what you\u0026rsquo;re looking for.\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eBut it goes further than that. Before the query even hits the vector database, it runs through a fuzzy normalization layer. Every country and location name in our system is cached at startup. When a customer types \u0026ldquo;Bali\u0026rdquo; we know they mean Indonesia. When they write \u0026ldquo;Tailand\u0026rdquo; we know what they meant. Aliases, spelling variations, language differences, all resolved before the search begins. Without this, you get empty results for queries that should have matched. And on top of that, the search combines semantic similarity with metadata filters. Country, duration, product type, these aren\u0026rsquo;t just fields in a database. They\u0026rsquo;re active filters that work together with the vector search to narrow results before the reranker even starts.\u003c/p\u003e\n\u003cp\u003eAfter reranking, the results split into two paths. The full set of reranked results pushes to the frontend immediately, but blurred. The customer sees product cards appearing while Joy is still thinking. It signals progress without overwhelming. Meanwhile, the LLM gets a stripped-down version of those results, only the fields it needs to reason about. This projection saves roughly a third of the tokens, which means faster responses and lower costs. Then Joy composes her answer and discusses the 4 or 5 trips that genuinely fit the question. Only those discussed products become visible to the customer. The rest disappear. What the customer sees is a curated, considered selection. Not a list of 10 search results. A recommendation of 5 that actually make sense.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eBroad fetch → Rerank → Project. Three steps that make the difference between a search result and a recommendation.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch3 id=\"the-memory-problem\"\u003eThe memory problem\u003c/h3\u003e\n\u003cp\u003eThis is the part that took the most iterations to get right.\u003c/p\u003e\n\u003cp\u003eConversational AI has a dirty secret: it doesn\u0026rsquo;t remember anything. Every LLM call is stateless. The model has no idea what was said before unless you explicitly feed it the context. And there\u0026rsquo;s a limit to how much context you can feed.\u003c/p\u003e\n\u003cp\u003eFor a simple Q\u0026amp;A chatbot, this doesn\u0026rsquo;t matter. Someone asks a question, gets an answer, done.\u003c/p\u003e\n\u003cp\u003eBut for a travel assistant? Memory is everything.\u003c/p\u003e\n\u003cp\u003eA customer says: \u0026ldquo;I want to go to Thailand for two weeks.\u0026rdquo; Three turns later: \u0026ldquo;Actually, make it three weeks. And I want to include Laos.\u0026rdquo; Five turns later: \u0026ldquo;What was that trip you showed me earlier with the cooking class?\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eWithout memory, every turn is a blank slate. The system doesn\u0026rsquo;t know about Thailand. Doesn\u0026rsquo;t know about the three-week change. Doesn\u0026rsquo;t know about Laos. Definitely doesn\u0026rsquo;t know about the cooking class trip from seven messages ago.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWithout memory, every message is a first date. Your AI has no idea what happened before.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThis wasn\u0026rsquo;t solved in one attempt. Not in two. Not even in three.\u003c/p\u003e\n\u003cp\u003eThe first version was simple: feed the last 10 question-answer pairs into the LLM context. It worked for short conversations. But our conversations aren\u0026rsquo;t short. Customers explore, compare, come back to earlier ideas, change their mind. Ten turns isn\u0026rsquo;t enough when someone has been chatting for twenty minutes about three different countries.\u003c/p\u003e\n\u003cp\u003eSo we tried generating summaries in the background. The idea was sound: condense the conversation into key points so the LLM always has the full picture without eating up the entire context window. The reality was different. The LLM started hallucinating details into the summaries. Mixing up countries. Attributing preferences the customer never mentioned. The summary was supposed to be the source of truth, and it was making things up.\u003c/p\u003e\n\u003cp\u003eThen we designed a three-stage memory system: short-term, ephemeral, and long-term. The ephemeral layer was supposed to track which topics belonged to which destination, which preferences applied to which part of the trip. On paper, elegant. In practice, a nightmare. It required the AI to understand complex relational context, like \u0026ldquo;when they said budget was flexible, they meant for Thailand, not for Vietnam.\u0026rdquo; It couldn\u0026rsquo;t. Too many edge cases. Too fragile.\u003c/p\u003e\n\u003cp\u003eWhat we landed on was simpler and more robust. Every message gets embedded as a vector in a dedicated memory collection in the same database we use for product search. When the customer says \u0026ldquo;that trip with the cooking class,\u0026rdquo; the system runs a semantic search across all past messages in that session. It finds the relevant message even if the exact words don\u0026rsquo;t match. Maybe Joy called it a \u0026ldquo;culinary experience\u0026rdquo; three turns ago. Vector search doesn\u0026rsquo;t care about word matching. It matches meaning. Combined with the last 10 exchanges in the prompt for immediate context, Joy doesn\u0026rsquo;t need to make that tool call for every question. She has the recent conversation right there, and the full history searchable when she needs it.\u003c/p\u003e\n\u003cp\u003eFour attempts. Each one taught us something about what AI can and can\u0026rsquo;t do with context. The final version works not because it\u0026rsquo;s the smartest design, but because it\u0026rsquo;s the most honest about the limitations.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAI memory isn\u0026rsquo;t one problem. It\u0026rsquo;s two: recalling what was said, and understanding what was meant. We needed two different systems to solve them.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch3 id=\"uiux-the-invisible-work\"\u003eUI/UX: the invisible work\u003c/h3\u003e\n\u003cp\u003eHere\u0026rsquo;s where most AI projects die, and they don\u0026rsquo;t even realize it.\u003c/p\u003e\n\u003cp\u003eYou can have the most sophisticated RAG pipeline in the world. The smartest retrieval. The best reranking. But if the interface feels clunky, slow, or confusing, none of it matters. The customer doesn\u0026rsquo;t care about your architecture. They care about how it feels.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eIf your AI is smart but your interface is slow, you built a genius locked in a closet.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eWe went deep on this. And \u0026ldquo;deep\u0026rdquo; means dozens of iterations on things most people would consider details. But details are the product.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eStreaming that feels natural.\u003c/strong\u003e LLM responses stream token by token over WebSocket. But raw streaming creates flickering text that\u0026rsquo;s jittery and unpleasant. We buffer at 50ms intervals. Fast enough to feel real-time, slow enough to prevent constant re-renders. The difference is subtle but massive in terms of perceived quality.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eThe speed tradeoff.\u003c/strong\u003e Let\u0026rsquo;s be honest about something: a product search with a full answer takes 20 to 30 seconds. That\u0026rsquo;s long. We know it\u0026rsquo;s long. But we made a deliberate choice: quality over speed. The broad fetch, the reranking, the semantic matching, every step in the pipeline exists because it makes the results better. Cut any of them and you get faster responses with worse recommendations. We\u0026rsquo;d rather have Joy work for 25 seconds and deliver 5 trips that genuinely fit than return 10 mediocre matches in 8 seconds.\u003c/p\u003e\n\u003cp\u003eSo the question became: how do we make 25 seconds not feel like 25 seconds? That\u0026rsquo;s where the UX work came in. Joy shows what she\u0026rsquo;s doing. Searching products. Looking up destination information. Checking conversation history. Every tool call is visible as it happens. Product cards push in blurred while she\u0026rsquo;s still composing her answer. Once the answer is ready, only the trips Joy actually discusses become visible. The rest fade away. The customer never stares at a blank screen. They see the process unfold. And it turns out that watching something work feels completely different from waiting for something to finish.\u003c/p\u003e\n\u003cp\u003ePart of the latency is also a language problem. Dutch is poorly represented in most embedding and language models. The semantic understanding that works beautifully in English degrades when your customers and your product data are in Dutch. It means more processing, more careful matching, more effort to get the same level of quality.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWhat is GPT-NL?\u003c/strong\u003e \u003ca href=\"https://gpt-nl.nl/\" data-external\u003eGPT-NL\u003c/a\u003e is a Dutch initiative to build a large language model specifically trained on and optimized for the Dutch language.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://gpt-nl.nl/\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eGPT-NL: Een verantwoord taalmodel voor Nederland\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eIn Nederland wordt gewerkt aan de ontwikkeling van een eigen open taalmodel genaamd GPT-NL. Wij gebruiken alleen data om het model te trainen als we die rechtmatig kunnen verkrijgen en bieden zo een verantwoord alternatief op bestaande LLMs. Dit model is essentieel voor het versterken en behouden van onze digitale soevereiniteit.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-technical-architecture-behind-a-conversational-travel-channel/bookmarks/01-icon_hu_6ebbb399eb63616a.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eGPT-NL\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-technical-architecture-behind-a-conversational-travel-channel/bookmarks/01-thumb_hu_6ec85365e1fe875f.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cp\u003eWhere current models like GPT and Claude treat Dutch as a non existent language, GPT-NL aims to make Dutch a first-class citizen in AI. For applications like ours, where both the customer and the product data are Dutch, a model that truly understands the language could significantly improve semantic search quality and reduce the processing overhead we currently need to compensate for that gap. We\u0026rsquo;re watching this project closely.\u003c/p\u003e\n\u003cp\u003eAnd then there\u0026rsquo;s model selection. The quality of the conversation depends heavily on which LLM you use. We run on Sonnet 4.6 as our primary, and the difference with smaller or cheaper models is not subtle. It\u0026rsquo;s the difference between a conversation that feels natural and one that feels scripted. Haiku, the faster and cheaper option, feels like a downgrade the moment you switch. GPT serves as our automatic fallback, but it\u0026rsquo;s a step back in conversational quality. These aren\u0026rsquo;t interchangeable commodities. The model you choose defines the experience your customer gets.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInstant page loads.\u003c/strong\u003e The entire conversation persists in local storage. When you refresh the page or come back later, your conversation is already there. Instantly. No loading spinner. No \u0026ldquo;reconnecting\u0026hellip;\u0026rdquo; message. The sync with the server happens silently in the background.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMarkdown normalization.\u003c/strong\u003e LLMs have opinions about formatting. Claude loves bold headers with emoji. GPT loves numbered lists. We normalize everything server-side. Bold-with-emoji gets converted to clean headers. Horizontal rules get stripped. Font weights get tuned down. The result looks like it was designed, not generated. The customer should never feel like they\u0026rsquo;re reading AI output.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSession continuity.\u003c/strong\u003e Your session survives across page refreshes, browser closes, and reconnections. Come back a day later, your conversation is still there. Your saved trips are still there. And if a session does expire, you get a clean overlay explaining what happened, not a broken page or a cryptic error.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eThe lead flow.\u003c/strong\u003e When a customer is ready to talk to a specialist, the system generates a travel proposal pre-filled with what it already knows from the conversation. Destinations discussed, trip preferences, products explored. Minimal friction. Maximum context. The human specialist picks up exactly where Joy left off. That\u0026rsquo;s not a handoff. That\u0026rsquo;s the whole point. Joy exists to make that human conversation better, not to avoid it.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eEvery one of these details took multiple iterations. Every one of them is invisible to the customer. That\u0026rsquo;s the point.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch3 id=\"security-an-honest-note\"\u003eSecurity: an honest note\u003c/h3\u003e\n\u003cp\u003eI\u0026rsquo;ll keep this section brief and honest. Security isn\u0026rsquo;t something you finish. It\u0026rsquo;s something you work on constantly.\u003c/p\u003e\n\u003cp\u003eWe have a dual guard system. An input guard before the LLM that catches prompt injection attempts, and an output guard after the LLM that catches credential leaks or system prompt fragments in responses. Both are deterministic and add virtually zero latency.\u003c/p\u003e\n\u003cp\u003eThe system prompt is hardened with anti-jailbreak instructions. There\u0026rsquo;s rate limiting, bot protection, and IP management for persistent abuse.\u003c/p\u003e\n\u003cp\u003eIs it bulletproof? No. Nothing is. But it\u0026rsquo;s built with defense in depth in mind, and we iterate on it as new attack patterns emerge. If you\u0026rsquo;re building any AI interface that talks to the public and you haven\u0026rsquo;t thought about input/output guards, you have a problem waiting to happen.\u003c/p\u003e\n\u003ch3 id=\"the-bigger-picture-an-extra-channel-not-a-replacement\"\u003eThe bigger picture: an extra channel, not a replacement\u003c/h3\u003e\n\u003cp\u003eLet me be very clear about what Joy is and what she isn\u0026rsquo;t.\u003c/p\u003e\n\u003cp\u003eJoy is not a replacement for our website. She\u0026rsquo;s not a replacement for phone or email. She\u0026rsquo;s an additional channel. A new way for customers to explore our products, alongside everything that already exists. She will never be a replacement for human contact. We value that the most in our company. Joy is a means to human contact. She helps customers figure out what they want, so that when they talk to one of our specialists, that conversation starts at a completely different level.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eJoy isn\u0026rsquo;t the destination. She\u0026rsquo;s the road that leads to a real conversation with a real person.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe MCP server is already live, which means AI assistants can already query our data programmatically. When the Web 4.0 vision materializes, when Lisa\u0026rsquo;s AI assistant autonomously searches for her perfect Thailand trip, the endpoint already exists. Not because we scrambled to build it. Because we\u0026rsquo;ve been exploring this infrastructure for over a year.\u003c/p\u003e\n\u003cp\u003eJoy isn\u0026rsquo;t the end product. She\u0026rsquo;s the beginning of a channel strategy that includes both human-to-AI and AI-to-AI interactions. And she\u0026rsquo;s a test, a very serious, very thorough test, of whether conversational AI can actually add value for travel customers today.\u003c/p\u003e\n\u003cp\u003eThe results so far have been better than I expected. Not flawless, nothing is, but genuinely impressive in how natural the conversations feel and how relevant the recommendations are. When you see a customer describe a vague idea and Joy comes back with trips that actually fit, you realize this isn\u0026rsquo;t a gimmick. It works. And it works because of everything underneath it.\u003c/p\u003e\n\u003ch3 id=\"so-what-are-we-trying-to-solve-again\"\u003eSo, what are we trying to solve again?\u003c/h3\u003e\n\u003cp\u003eI opened with this question because it\u0026rsquo;s the one that guided everything we built. Not \u0026ldquo;how do we use AI?\u0026rdquo; Not \u0026ldquo;what\u0026rsquo;s our competitors doing?\u0026rdquo; Not \u0026ldquo;how do we look innovative?\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eJust: what problem are we solving, and does this solution actually work?\u003c/p\u003e\n\u003cp\u003eWe built Joy to find out. With real architecture. Real data. Real customers. And the willingness to say \u0026ldquo;it\u0026rsquo;s not ready yet\u0026rdquo; if that turned out to be the case.\u003c/p\u003e\n\u003cp\u003eIf you\u0026rsquo;re thinking about adding AI to your product, start there. Not with the tool. Not with the vendor pitch. Not with the pressure to keep up.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eStart with the question. Build from the answer.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eYour customers deserve better than a FAQ with a face. They deserve a real conversation. And eventually, a real person. Make sure your AI leads them there, not away from it.\u003c/p\u003e\n\u003cp\u003eIf you haven\u0026rsquo;t read \u003ca href=\"https://www.allsubjectsmatter.nl/can-a-conversation-be-a-travel-channel-we-built-the-architecture-to-find-out/\" data-preview\u003ePart 1\u003c/a\u003e, that\u0026rsquo;s where the story starts: why we built this, what we learned from failing at it in 2019, and why you can\u0026rsquo;t get here with a plugin.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/can-a-conversation-be-a-travel-channel-we-built-the-architecture-to-find-out/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eCan a conversation be a travel channel? Part 1: We built one to find out.\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eWe built a full AI travel interface from scratch, not to say we do AI, but to test whether a conversation can actually be a meaningful channel for personalized travel. This is what it took, why a plugin will never get you there, and why most AI chatbots are just expensive search bars.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-technical-architecture-behind-a-conversational-travel-channel/bookmarks/02-icon_hu_8a3f1fd6616d7152.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco Loup.\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-technical-architecture-behind-a-conversational-travel-channel/bookmarks/02-thumb_hu_bbbba9d6b5c5a7aa.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n","date_modified":"2026-03-11T15:47:46+01:00","date_published":"2026-03-11T09:54:10+01:00","id":"https://www.allsubjectsmatter.nl/the-technical-architecture-behind-a-conversational-travel-channel/","language":"en","summary":"The technical deep-dive into how we built Joy: a RAG pipeline with vector search and reranking, a memory system that took four iterations, and the UI/UX decisions that make a 25-second response feel more instant.","tags":["Building"],"title":"Can an AI conversation be a travel channel? Part 2: The technical architecture","url":"https://www.allsubjectsmatter.nl/the-technical-architecture-behind-a-conversational-travel-channel/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eWe built an AI-powered travel interface from scratch. Not a chatbot. Not a widget. Not a GPT-wrapper with a smiley face bolted onto a website. A full conversational interface with its own vector database, its own retrieval pipeline, its own memory system, and real-time streaming over WebSockets. Not to replace anything. To test whether conversational AI is actually ready to be a meaningful channel for travel. Here\u0026rsquo;s what we found, what it took, and why we think most of what\u0026rsquo;s out there right now is noise.\u003c/p\u003e\n\u003chr\u003e\n\u003cp\u003eThis is a two-part story. This first part covers the why: what problem we\u0026rsquo;re solving, what we learned from failing at this before, and why a plugin will never get you here. \u003ca href=\"https://www.allsubjectsmatter.nl/the-technical-architecture-behind-a-conversational-travel-channel/\" data-preview\u003ePart 2\u003c/a\u003e goes deeper on the architecture: the RAG pipeline, the memory system, the UI/UX decisions, and the technical tradeoffs we made.\u003c/p\u003e\n\u003ch2 id=\"what-are-you-actually-trying-to-solve\"\u003eWhat are you actually trying to solve?\u003c/h2\u003e\n\u003cp\u003eThat\u0026rsquo;s the question I keep coming back to. Every time I see a company announce their new \u0026ldquo;AI-powered\u0026rdquo; feature. Every time someone asks me what we\u0026rsquo;re doing with AI. Every time a vendor pitches yet another plugin.\u003c/p\u003e\n\u003cp\u003eIt\u0026rsquo;s the most important question in tech right now. And almost nobody is asking it.\u003c/p\u003e\n\u003cp\u003eInstead, the entire industry is working backwards: starting with \u0026ldquo;we need to do something with AI\u0026rdquo; and then looking for a place to put it. Companies hear \u0026ldquo;AI\u0026rdquo; and they hear \u0026ldquo;innovation\u0026rdquo; and they think they\u0026rsquo;re falling behind if they don\u0026rsquo;t do something right now.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe pressure to \u0026ldquo;do something with AI\u0026rdquo; has become louder than the question of what are you trying to achieve.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eAnd so what happens? Companies reach for the most visible, most marketable application they can find: a chatbot. Slap it on the website. Give it a name and a friendly face. Connect it to a knowledge base. Done. \u0026ldquo;We do AI now.\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eHere\u0026rsquo;s the problem: the integration is always limited. Some connect to a product database. Some pull live availability. But as a plugin, there\u0026rsquo;s a ceiling to how deep that integration goes. It\u0026rsquo;s always working with whatever data it gets access to, in whatever format the API provides, with whatever context the vendor decided was enough. The result is an interaction that looks conversational on the surface but is fundamentally constrained in what it can actually do.\u003c/p\u003e\n\u003cp\u003eAnd because it\u0026rsquo;s a plugin, it can never be fully seamless. It will always be a separate layer on top of your website, not part of it. A different design language. A different interaction model. A different experience. The customer feels it, even if they can\u0026rsquo;t articulate it. It\u0026rsquo;s the difference between a room that was designed as a whole and a room where someone placed a kiosk in the corner.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eA smarter gatekeeper is still a gatekeeper. Your customers wanted a door.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eWe\u0026rsquo;ve been down this road ourselves. Back in 2019, we built the \u003ca href=\"https://www.allsubjectsmatter.nl/how-333travel-is-revolutionizing-the-travel-industry/\" data-preview\u003efirst version of Joy\u003c/a\u003e on Flow.ai. An intent-based chatbot that could understand natural language, extract travel preferences from a single sentence, and generate leads for our specialists. It worked. Technically. But customer engagement was too low to justify keeping it alive. We retired Joy in early 2022.\u003c/p\u003e\n\u003cp\u003eThe lesson wasn\u0026rsquo;t that the technology was bad. The lesson was that people have learned to distrust chatbots. And rightfully so. For years, companies have used chatbots as a barrier to contact, even when they claim otherwise. Customers know this. They\u0026rsquo;ve experienced it. After three wrong answers, you don\u0026rsquo;t think \u0026ldquo;let me try again.\u0026rdquo; You think \u0026ldquo;how do I get to a real person?\u0026rdquo; Some people don\u0026rsquo;t even bother engaging. They see a chat bubble with a face and their immediate reaction is to close it. Not because they\u0026rsquo;re impatient, but because experience taught them it\u0026rsquo;s a waste of time.\u003c/p\u003e\n\u003cp\u003eThat\u0026rsquo;s not a technology problem. That\u0026rsquo;s a trust problem. And it\u0026rsquo;s one you can\u0026rsquo;t solve by making the same format smarter.\u003c/p\u003e\n\u003cp\u003eOn top of that, the interface of a chatbot plugin is simply limiting. A small window in the corner of your screen. A few lines of text. Maybe a button or two. You can\u0026rsquo;t show rich product cards in that. You can\u0026rsquo;t display a map of a trip. You can\u0026rsquo;t let someone browse, compare, save trips for later. The format dictates the experience, and the format is a box.\u003c/p\u003e\n\u003cp\u003eThat experience is exactly why we didn\u0026rsquo;t build another chatbot. Not a smarter one. Not a better one. Something fundamentally different. A full conversational interface, built from scratch, designed to inspire. Where someone can explore destinations, discover trips they didn\u0026rsquo;t know existed, and when they\u0026rsquo;re ready, transition into contact with a specialist after a conversation that actually felt good. Not after three failed attempts to get past a gatekeeper.\u003c/p\u003e\n\u003cp\u003eThe real question was never \u0026ldquo;how do we add AI to our website?\u0026rdquo; The real question is: where does AI genuinely create value that wasn\u0026rsquo;t possible before? That\u0026rsquo;s what we set out to test.\u003c/p\u003e\n\u003ch2 id=\"why-i-built-what-i-built\"\u003eWhy I built what I built\u003c/h2\u003e\n\u003cp\u003eI didn\u0026rsquo;t build this because I think websites are dead or because I wanted to say we \u0026ldquo;do something with AI.\u0026rdquo; I\u0026rsquo;ve written before about \u003ca href=\"https://www.allsubjectsmatter.nl/lets-stop-talking-about-ai-and-start-solving-real-problems/\" data-preview\u003ehow much I despise that mentality\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eI built this because of a question I\u0026rsquo;ve been testing for over a year now: is conversational AI truly ready to be a real channel?\u003c/p\u003e\n\u003cp\u003eIn my \u003ca href=\"https://www.allsubjectsmatter.nl/web-4-0-and-the-autonomous-web-the-rise-of-ai-driven-customers/\" data-preview\u003eWeb 4.0 post\u003c/a\u003e, I wrote about the autonomous web, a world where AI assistants don\u0026rsquo;t just retrieve information but act as independent digital consumers on behalf of their humans. Where Lisa tells her AI assistant \u0026ldquo;find me a two-week trip to Thailand\u0026rdquo; and the AI contacts travel companies through MCP-enabled interfaces, negotiates, refines, and presents options. Without Lisa ever opening a browser.\u003c/p\u003e\n\u003cp\u003eThat might sound like removing humans from the equation. It\u0026rsquo;s not. The technology enables full autonomy. But that doesn\u0026rsquo;t mean we want to remove the human. It means the customer gets to choose. And our default will always be: lead to a person. More on that later.\u003c/p\u003e\n\u003cp\u003eThat future is coming. Maybe not tomorrow, but it\u0026rsquo;s coming. And if you\u0026rsquo;re a travel company that takes itself seriously, you\u0026rsquo;d better be ready for it.\u003c/p\u003e\n\u003cp\u003eSo I built Joy, an AI travel assistant for 333travel, not as a product launch, not as a marketing play, but as a serious test. Can we build a conversational interface that actually adds value? That understands our products? That doesn\u0026rsquo;t hallucinate as much? That creates a genuinely useful experience as an additional channel next to our website, phone, and email?\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eI didn\u0026rsquo;t build this to replace our website. I built this to answer a question: can a conversation be a meaningful channel for personalized travel?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"what-this-actually-is\"\u003eWhat this actually is\u003c/h2\u003e\n\u003cp\u003eLet me be specific about what Joy does:\u003c/p\u003e\n\u003cp\u003eJoy sits on top of our entire product database. Every roundtrip, every hotel, every excursion, every destination, all embedded as vectors in a dedicated database. When a customer says \u0026ldquo;I want something adventurous in Southeast Asia, around two weeks, not too touristy,\u0026rdquo; Joy doesn\u0026rsquo;t keyword-match against a PDF. She searches semantically across hundreds of products, reranks results based on actual relevance, and responds with options that genuinely fit the intent.\u003c/p\u003e\n\u003cp\u003eAnd she remembers the conversation. Not just the last message, the whole thread. She knows what trips were discussed, what was saved, what was dismissed. When you say \u0026ldquo;that trip you mentioned earlier with the cooking class,\u0026rdquo; she understands what you mean even if she originally called it a \u0026ldquo;culinary experience.\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eBut here\u0026rsquo;s the key difference with any chatbot out there: she doesn\u0026rsquo;t just know travel in general. She knows \u003cem\u003eour\u003c/em\u003e travel. Our specific products. Our specific itineraries. Our destinations that our team personally visited. That\u0026rsquo;s not something you can easily bolt on with a plugin, no matter how smart the LLM behind it is.\u003c/p\u003e\n\u003cp\u003eAnd the moment you want human contact? Joy hands the full context to a real travel specialist who knows exactly what you\u0026rsquo;ve been exploring. No \u0026ldquo;please describe your inquiry again.\u0026rdquo; No cold transfer. Full continuity.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eJoy doesn\u0026rsquo;t replace conversations with specialists. She makes them better.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eWhat she doesn\u0026rsquo;t do (yet): she doesn\u0026rsquo;t handle customer service queries, airline information, or post-booking support. This is purely a product discovery and inspiration channel. We\u0026rsquo;ll get there. But we\u0026rsquo;re not going to pretend she does things she doesn\u0026rsquo;t. And honestly, when someone reaches out to customer service, there\u0026rsquo;s already a problem. That\u0026rsquo;s the worst possible moment to put a machine between you and your customer. That\u0026rsquo;s when you show up. Personally. So we are still debating about what we want her to do there. The nuance is always in the details.\u003c/p\u003e\n\u003ch2 id=\"why-you-cant-get-here-with-a-plugin\"\u003eWhy you can\u0026rsquo;t get here with a plugin\u003c/h2\u003e\n\u003cp\u003eYou cannot achieve what we built by installing a plugin. Not with ChatGPT. Not with any off-the-shelf \u0026ldquo;AI widget.\u0026rdquo; Not with any SaaS tool that promises to \u0026ldquo;make your website smarter in minutes.\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eAnd it\u0026rsquo;s not because the AI models aren\u0026rsquo;t good enough. The models are fine. The problem is everything around them.\u003c/p\u003e\n\u003cp\u003eThe entire system depends on data that we own, structure, and control. Every trip in our database has been visited by our own team. Every itinerary is our own creation. Every piece of destination knowledge lives in our own systems. This isn\u0026rsquo;t data we license from a third party or pull from a brochure catalog.\u003c/p\u003e\n\u003cp\u003eThat means we can embed it. We can structure it exactly how we need it for semantic search. We can update it the moment something changes. We control the full pipeline from raw product data to vector embedding to retrieval to response.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eYou can\u0026rsquo;t build a smart interface on top of piles of dumb data.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIf your product data lives in someone else\u0026rsquo;s system, if you\u0026rsquo;re reselling trips from a catalog, if your \u0026ldquo;database\u0026rdquo; is a collection of PDFs from tour operators, you\u0026rsquo;re at the mercy of whatever structure someone else decided on. You can\u0026rsquo;t build intelligence on top of data you don\u0026rsquo;t understand or control.\u003c/p\u003e\n\u003cp\u003eAnd many companies aren\u0026rsquo;t just starting from zero, they\u0026rsquo;re starting from a deficit. Their existing systems are already inadequate. The booking engine is outdated. The product data is scattered across spreadsheets and PDFs. The customer journey has gaps everywhere. And instead of fixing those fundamentals, they think it is smart to layer an AI chatbot on top of it. As if intelligence on top of dysfunction creates something functional. It doesn\u0026rsquo;t. It just makes the dysfunction harder to diagnose.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eYou\u0026rsquo;re not solving a problem. You\u0026rsquo;re decorating one.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eBuilding what we built required three things:\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eOwning our data.\u003c/strong\u003e Every trip is ours. Visited by our team. Structured in our database. Embedded in our vector store. You can\u0026rsquo;t build semantic search over data you don\u0026rsquo;t structure yourself.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eOwning our systems.\u003c/strong\u003e Our own backend, frontend, deployment pipeline. When we needed a dual reranker, we built it. When we needed semantic conversation memory, we built it. When we needed an MCP server for Web 4.0 readiness, we built it. No vendor approval needed. No feature requests. No roadmap dependencies.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eOwning the entire chain.\u003c/strong\u003e From the moment a customer opens the chat to the moment they\u0026rsquo;re talking to a specialist, every step is ours. The AI knows our products because we made the products. The specialist knows the conversation because our system feeds it to them.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe AI is the last 10% of the work. The first 90% is building a company that can support it.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIn \u003ca href=\"https://www.allsubjectsmatter.nl/the-technical-architecture-behind-a-conversational-travel-channel/\" data-preview\u003ePart 2\u003c/a\u003e, I\u0026rsquo;ll show you how it is actually running under the hood. The architecture, the search pipeline, the memory system that took many attempts to get right, and why 25 seconds of waiting can feel like 5 when you design it right.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/the-technical-architecture-behind-a-conversational-travel-channel/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eCan a conversation be a travel channel? Part 2: The technical architecture\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eThe technical deep-dive into how we built Joy: a RAG pipeline with vector search and reranking, a memory system that took four iterations, and the UI/UX decisions that make a 25-second response feel instant.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/can-a-conversation-be-a-travel-channel-we-built-the-architecture-to-find-out/bookmarks/01-icon_hu_8a3f1fd6616d7152.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco Loup.\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/can-a-conversation-be-a-travel-channel-we-built-the-architecture-to-find-out/bookmarks/01-thumb_hu_cb9ae8a41b04a42d.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n","date_modified":"2026-03-11T15:47:56+01:00","date_published":"2026-03-10T13:46:34+01:00","id":"https://www.allsubjectsmatter.nl/can-a-conversation-be-a-travel-channel-we-built-the-architecture-to-find-out/","language":"en","summary":"We built a full AI travel interface from scratch, not to say we do AI, but to test whether a conversation can actually be a meaningful channel for personalized travel. This is what it took, why a plugin will never get you there, and why most AI chatbots are just expensive search bars.","tags":["Building","Strategy"],"title":"Can an AI conversation be a travel channel? Part 1: We built one to find out.","url":"https://www.allsubjectsmatter.nl/can-a-conversation-be-a-travel-channel-we-built-the-architecture-to-find-out/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eI built a \u0026ldquo;production\u0026rdquo; app using Vibecoding. 97,089 lines generated. Only 25,056 made it to production. Four complete refactors. The AI didn\u0026rsquo;t fail because of bad prompts. It failed because AI forgets, loses context, and creates duplicate logic no matter how well you guide it. Vibecoding isn\u0026rsquo;t about letting AI build for you. It\u0026rsquo;s about knowing your system well enough to filter the noise. Without that knowledge, you\u0026rsquo;re only shipping guesses.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"what-vibecoding-actually-means\"\u003e\u003cstrong\u003eWhat Vibecoding actually means\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003ePeople think vibecoding means letting AI build your app while you watch. Reality check kicking in here, this is not at all the case if you need an actual working product with the expected result every time.\u003c/p\u003e\n\u003cp\u003eVibecoding still means you take control. You prompt with precision. You experiment. You guide. You filter. It\u0026rsquo;s still hands-on work, just with different leverage.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003e\u0026ldquo;A fool with a tool is still a fool\u0026rdquo;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eAnd if you don\u0026rsquo;t know what you\u0026rsquo;re doing? The result is the same as it always was: a mess. Vibecoding without the right knowledge is just plain wrong. And the main problem with AI, is that it keeps producing code after code after code. It just does what you ask. it isn\u0026rsquo;t critical about that. So if you don\u0026rsquo;t know what you are doing you are contradicting yourself all the time with programmed functionality that never works together within your app.\u003c/p\u003e\n\u003cp\u003eAi is king in fallbacks. I would have had 5 different fallbacks for every api call that fails. Shouldn\u0026rsquo;t we just make the right way work instead of making fallbacks. AI is also lazy, it rather suppresses notices then fixing them. especially if it won\u0026rsquo;t work on the first try.\u003c/p\u003e\n\u003ch2 id=\"i-built-a-production-app-with-ai-but-not-because-of-ai\"\u003e\u003cstrong\u003eI built a production app with AI (but not because of AI)\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003eI used Windsurf to build an entire application. No human developers. No code written by me directly. we use Sonnet 4.5 as our knowledge.\u003c/p\u003e\n\u003cp\u003eThe result after a lot of frustration: a full production game with backend logic, login flows, async data writes, mail queues, admin security, monitoring, and cartesian algorithms that update based on user interaction.\u003c/p\u003e\n\u003cp\u003eThe AI wrote all of it. I haven\u0026rsquo;t touched a single line of code. So even the simples css change. I let AI do it.\u003c/p\u003e\n\u003cp\u003eBut here\u0026rsquo;s what nobody tells often: I had to completely refactor the entire project four times. And not because I prompted poorly. I used KISS principles, DRY standards, PSR guidelines. We wrote decisions and created implementation documents on forehand. We used guidelines for the languages that we used.\u003c/p\u003e\n\u003cp\u003eIt didn\u0026rsquo;t matter in the end.\u003c/p\u003e\n\u003cp\u003eAI forgets. It loses context. It starts creating duplicate functions without realizing it already wrote that logic three files ago. The more complex your project gets, the more it drifts. And the more it isn\u0026rsquo;t done right in the first attempt AI thinks it needs to maintain backwards compatibility, complicating your code even more. No matter how you prompt it the first time or with project rules.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eFour complete refactors later, I learned: AI doesn\u0026rsquo;t forget context. It never had it ;).\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe only reason the final version works is because I knew exactly what I was building. I had the system in my head. I could spot when AI was repeating itself. I knew when to stop it, when to redirect it, when to start over. I checked every file for duplicate fallback data. duplicates and so on. I also used the very usefull npm package knip\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://www.npmjs.com/package/knip\" data-external\u003ehttps://www.npmjs.com/package/knip\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eYou can\u0026rsquo;t vibecode something into production without understanding your product. Period. And you need to accept that bug fixing is relying on your AI to fix it without breaking anything else, Good luck with that\u0026hellip;\u003c/p\u003e\n\u003ch2 id=\"the-data-tells-the-rest-of-our-story\"\u003e\u003cstrong\u003eThe data tells the rest of our story\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003eDuring our Vibecoding, Windsurf produced:\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/i-vibecoded-an-app-and-this-is-what-i-learned/images/Screenshot-2025-11-10-at-09.28.05_hu_8c902f8cc6fdc18d.webp\" alt=\"\" width=\"560\" height=\"442\" data-zoom-src=\"https://www.allsubjectsmatter.nl/i-vibecoded-an-app-and-this-is-what-i-learned/images/Screenshot-2025-11-10-at-09.28.05_hu_f69c6e1b7aae2732.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003cul\u003e\n\u003cli\u003e97,089 lines of code generated\u003c/li\u003e\n\u003cli\u003e2,925 prompts sent\u003c/li\u003e\n\u003cli\u003e5,693 credits used\u003c/li\u003e\n\u003cli\u003e15,734 tool calls (including 7,000 code edits)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eWhat actually made it into production:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e25,056 lines of code\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eOver 70,000 lines got scrapped, replaced or refactored.\u003c/p\u003e\n\u003cp\u003eNow, you might think this means you just need to \u0026ldquo;guide AI better\u0026rdquo; or \u0026ldquo;prompt more precisely.\u0026rdquo; That\u0026rsquo;s not the point I wanted to make.\u003c/p\u003e\n\u003cp\u003eThe point is: \u003cstrong\u003eyou can\u0026rsquo;t prevent AI spew right now.\u003c/strong\u003e No matter how well you prompt. No matter how structured your approach is. AI will overproduce. It will generate redundant code. It will lose track of what it already built. It will create noise. AI is build to always post a form of AI Verbal Diarrhea.\u003c/p\u003e\n\u003cp\u003eThat 70k+ lines of wasted output? That\u0026rsquo;s not a bug. That\u0026rsquo;s the current state of Vibecoding.\u003c/p\u003e\n\u003cp\u003eAnd if you don\u0026rsquo;t know your system well enough to recognize what\u0026rsquo;s garbage and what\u0026rsquo;s not, you\u0026rsquo;ll ship all of it and never be able to debug even with AI. Or worse, you\u0026rsquo;ll spend personally weeks debugging a codebase you don\u0026rsquo;t understand.\u003c/p\u003e\n\u003cp\u003eThe costs per 1000 credits are around 35 dollar. The development of this game cost me 210 dollar. Every company delivering Large Language Models is losing money. the code I produced should easily have costed 210 x 10 =2.100 dollar at least. I even wonder if they then even made money then.\u003c/p\u003e\n\u003cp\u003eApart from the money, is this really worth our precious electricity? I am wondering if you pay the real price, how feasible this will be.\u003c/p\u003e\n\u003ch2 id=\"what-people-get-wrong\"\u003e\u003cstrong\u003eWhat people get wrong\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003eAI isn\u0026rsquo;t magic. It\u0026rsquo;s a \u0026ldquo;smart\u0026rdquo;, fast assistant. But it doesn\u0026rsquo;t understand what matters to your business. It doesn\u0026rsquo;t grasp edge cases, constraints, risks, performance implications, or security vulnerabilities.\u003c/p\u003e\n\u003cp\u003eThat\u0026rsquo;s your job!\u003c/p\u003e\n\u003cp\u003eIn my case, AI helped build something real. But the AI didn\u0026rsquo;t secure the admin panel. It didn\u0026rsquo;t design the mail queue logic. It didn\u0026rsquo;t decide when to debounce async writes. I did. By being critical and asking the right questions due to build experience in the past.\u003c/p\u003e\n\u003cp\u003eWithout that direction, the entire thing would have collapsed.\u003c/p\u003e\n\u003cp\u003eAnd I see this pattern everywhere. Teams building AI products without ownership or direction, just to say they\u0026rsquo;ve \u0026ldquo;done something with AI.\u0026rdquo; It\u0026rsquo;s shallow. It\u0026rsquo;s fragile. It doesn\u0026rsquo;t scale.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;You\u0026rsquo;re not building products. You\u0026rsquo;re laundering prompts into deployments\u0026rdquo;\u003c/em\u003e\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eHow many times don\u0026rsquo;t you hear from higher hand: We need to do something with AI. We are falling behind. Behind of what?\u003c/p\u003e\n\u003ch2 id=\"the-customer-service-trap\"\u003e\u003cstrong\u003eThe customer service trap\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003eThen there\u0026rsquo;s also the AI customer service hype. Companies claim it\u0026rsquo;s perfect for 24/7 support, faster responses, fewer humans needed.\u003c/p\u003e\n\u003cp\u003eBut if you need a bot to talk to your customers, your real problem is probably somewhere else. AI doesn\u0026rsquo;t fix broken processes. It hides them. Temporarily.\u003c/p\u003e\n\u003cp\u003eIf you think less human contact is a win, you\u0026rsquo;ve already lost. If replying to questions takes a long time you might have to look at other things why people have so many questions and improve on that first.\u003c/p\u003e\n\u003ch2 id=\"the-actual-point\"\u003e\u003cstrong\u003eThe actual point\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003eVibecoding is just another tool. That\u0026rsquo;s all it is. A very cool tool in the right hands tho, but still a tool. And mainly good for prototyping, but with a good dosis of knowledge you can maybe produce standalone apps that actually work, and are at least testable by you and the outcome is traceable to the code written in a transparant way.\u003c/p\u003e\n\u003cp\u003eIt doesn\u0026rsquo;t replace thinking. It should amplify it, and it shouldn\u0026rsquo;t make you think less.\u003c/p\u003e\n\u003cp\u003eUsed right, it can be powerful. But you still need to know what you want to build. You need to own the structure. You need to filter the noise. You need to recognize AI spew when you see it. You need to lead the process.\u003c/p\u003e\n\u003cp\u003eOtherwise, you\u0026rsquo;re not building a prototype. You\u0026rsquo;re just generating guesses.\u003c/p\u003e\n\u003cp\u003eThe future isn\u0026rsquo;t AI-first. The companies that matter stay human-first. They understand the problem before picking the tool. And they keep asking the only question that counts before reaching out for AI to help them:\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWhat are we actually trying to solve?\u003c/strong\u003e\u003c/p\u003e\n","date_modified":"2025-11-10T11:50:09+01:00","date_published":"2025-11-10T11:37:31+01:00","id":"https://www.allsubjectsmatter.nl/i-vibecoded-an-app-and-this-is-what-i-learned/","language":"en","summary":"I built a \"production\" app using Vibecoding. 97,089 lines generated. Only 25,056 made it to production. Four complete refactors. The AI didn't fail because of bad prompts. It failed because AI forgets, loses context, and creates duplicate logic no matter how well you guide it.","tags":["Building"],"title":"I Vibecoded an app and this is what I learned.","url":"https://www.allsubjectsmatter.nl/i-vibecoded-an-app-and-this-is-what-i-learned/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eArtificial Intelligence is here to stay, that’s not the topic here.\u003c/p\u003e\n\u003cp\u003eThe problem is that \u003cstrong\u003eeveryone’s talking about AI\u003c/strong\u003e, but few are actually thinking clearly about how to really use it. Somewhere along the way, “we use AI” became a marketing tagline, not a technical decision. Everyone wants to be part of the AI wave otherwise, you don’t belong. If you have to believe big tech the way they are forcing it down everyone\u0026rsquo;s throat.\u003c/p\u003e\n\u003cp\u003eBut when you take a small step back, you’ll notice something strange:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eCompanies are solving problems with AI that they shouldn’t have in the first place.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003ePoor system designs, missing ownership, and spaghetti infrastructures are now being \u0026ldquo;fixed\u0026rdquo; by throwing LLMs on top of this pile of shit. And as a result, systems are becoming even harder to maintain, more expensive, and harder to reason about.\u003c/p\u003e\n\u003cp\u003eThis blog isn’t about tearing down AI.\u003cbr\u003e\nIt’s about \u003cstrong\u003eclear thinking, technical ownership\u003c/strong\u003e, and using the right tools for the job.\u003c/p\u003e\n\u003ch2 id=\"openai-investigation-research-paper\"\u003eOpenAI investigation research paper\u003c/h2\u003e\n\u003cp\u003eRecent research by OpenAI and Georgia Tech highlights a core truth that often gets ignored: \u003cstrong\u003eAI hallucinations, those moments where models make things up aren’t bugs. They’re a mathematical consequence of how these systems are built.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEven with perfect training data, the current architecture of generative AI rewards confident answers over admitting uncertainty. That means hallucinations are not only common, they’re \u003cem\u003einevitable\u003c/em\u003e, unless we fundamentally change how we train and evaluate these models.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAs the researchers put it: “Hallucinations are mathematically inevitable.” (\u003ca href=\"https://arxiv.org/pdf/2509.04664\" data-external\u003esource\u003c/a\u003e)\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"how-people-abuse-ai-to-solve-the-wrong-problems\"\u003eHow people abuse AI to solve the wrong problems.\u003c/h2\u003e\n\u003cp\u003eLet’s take a step back.\u003c/p\u003e\n\u003cp\u003eIf you’re paying $1,200/month for GPT-4 to parse email text, extract phone numbers, reformat JSON, and uppercase strings… you’re not doing innovation. You’re outsourcing common sense to an expensive black box.\u003c/p\u003e\n\u003cp\u003eYes, someone recently did that, and realized the cost effect too late. They then moved to GPT-4o-mini and brought the bill down to $200\u0026hellip;. But even then it still is a shit solution:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThose tasks didn’t need AI at all. Just basic software skills.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThis isn’t an isolated case. It’s happening to much sadly.\u003c/p\u003e\n\u003cp\u003ePeople assume AI is their fix to everything, while it only shows the persons incompetence because:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eIt increases costs\u003c/li\u003e\n\u003cli\u003eObscures what\u0026rsquo;s actually happening\u003c/li\u003e\n\u003cli\u003eIntroduces hidden dependencies\u003c/li\u003e\n\u003cli\u003eMakes systems almost impossible to debug and maintain\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eAI is used to \u003cem\u003ecover up\u003c/em\u003e a lack of system design, not to improve it.\u003c/p\u003e\n\u003ch2 id=\"ai-as-a-natural-language-interface-powerful-but-misused\"\u003eAI as a Natural Language interface, powerful but misused\u003c/h2\u003e\n\u003cp\u003eBut let’s be fair: LLMs truely are powerful when used right.\u003cbr\u003e\nThey allow you to describe what you want in natural language, and get a result, without understanding the underlying logic.\u003c/p\u003e\n\u003cp\u003eThis makes AI a fantastic tool for:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ePrototyping\u003c/li\u003e\n\u003cli\u003eInteracting with large knowledge bases\u003c/li\u003e\n\u003cli\u003eSummarizing unstructured data\u003c/li\u003e\n\u003cli\u003ePerforming fuzzy matching at scale\u003c/li\u003e\n\u003c/ul\u003e\n\u003cblockquote\u003e\n\u003cp\u003eLLMs are essentially a new kind of interface, just like the command line, the GUI, or APIs were before.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eBut they’re not \u003cem\u003emagic\u003c/em\u003e. And they are definitely not for free. And they’re almost always not the best way to solve a problem.\u003c/p\u003e\n\u003cp\u003eThe more critical the function, the more important it becomes to understand:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eWhat is being executed?\u003c/li\u003e\n\u003cli\u003eWhere is the data flowing?\u003c/li\u003e\n\u003cli\u003eWho owns the outputs?\u003c/li\u003e\n\u003cli\u003eWhat are the real costs?\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"conversational-ai-the-modern-chatbot-but-still-a-chatbot\"\u003eConversational AI, the modern chatbot, but still a chatbot\u003c/h2\u003e\n\u003cp\u003eThere \u003cem\u003eis\u003c/em\u003e a path forward for conversational AI. It’s just not the revolutionary leap people think it is with self thinking systems that wont hallucinate and always provide the right answer.\u003c/p\u003e\n\u003cp\u003eYes, LLMs can provide smarter, more human-like responses then before LLM\u0026rsquo;s became mainstream.\u003cbr\u003e\nYes, they can automate support flows that were previously a lot more complex to script.\u003c/p\u003e\n\u003cp\u003eBut let’s not fool ourselves:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003ePeople still want \u003cstrong\u003ereal, human contact\u003c/strong\u003e with your business especially when something goes wrong.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIf you outsource your entire customer service to an AI assistant, you\u0026rsquo;re sending a clear message (but the wrong one in my opinion) :\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u0026ldquo;We don’t care enough to talk to you ourselves.\u0026rdquo;\u003c/li\u003e\n\u003cli\u003e\u0026ldquo;You\u0026rsquo;re not worth the time of a real person.\u0026rdquo;\u003c/li\u003e\n\u003cli\u003e\u0026ldquo;We only sell our product. dont come complaining to us please.\u0026rdquo;\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThis probably saves costs in the short term.\u003cbr\u003e\nBut long-term? You’re destroying the trust with your brand. Especially because people typically contact a customer service when:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eThey’re frustrated\u003c/li\u003e\n\u003cli\u003eThey don’t understand something\u003c/li\u003e\n\u003cli\u003eThey have a complaint\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThat is \u003cstrong\u003enot\u003c/strong\u003e the time to offer them a chatbot, no matter how “smart” it sounds.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eGreat customer service is a relationship, not a cost center.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eSo yes, use conversational AI to assist your teams. But don’t let it replace human contact where empathy is needed. Dont make AI the interface of talking with your company for customers if you care about your product.\u003c/p\u003e\n\u003ch2 id=\"what-you-should-be-using-ai-for\"\u003eWhat you should be using AI for\u003c/h2\u003e\n\u003cp\u003eLet’s flip it around in this section. What \u003cem\u003eis\u003c/em\u003e a good use case for AI?\u003c/p\u003e\n\u003cp\u003eYou should use AI when:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eIt helps you do things that were previously too expensive, too slow, or technically too complex\u003c/li\u003e\n\u003cli\u003eYou’ve already optimized your architecture, but still have edge cases worth automating\u003c/li\u003e\n\u003cli\u003eYou\u0026rsquo;re enriching processes, not hiding technical debt\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eFor example:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ePrototyping difficult functionalities so you can test outcomes quicker\u003c/li\u003e\n\u003cli\u003eEnhancing product search with semantic understanding\u003c/li\u003e\n\u003cli\u003eIf you can make the hallucination visible in data they are perfect for BI (business intelligence)\u003c/li\u003e\n\u003cli\u003eGenerating tailored summaries for large internal reports\u003c/li\u003e\n\u003cli\u003eTranscribing phone calls and make summaries\u003c/li\u003e\n\u003cli\u003eImage to text so you can better semantic search on images in large image databases.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThese are tasks where AI really shines, because they were even more expensive, hard or impossible to solve in traditional systems.\u003c/p\u003e\n\u003cp\u003eBut again: \u003cstrong\u003ealways weigh the costs\u003c/strong\u003e.\u003cbr\u003e\nNot just in dollars, but in \u003cstrong\u003eclarity\u003c/strong\u003e, \u003cstrong\u003econtrol\u003c/strong\u003e, and \u003cstrong\u003ecustomer experience\u003c/strong\u003e. Always try to solve it without using AI first.\u003c/p\u003e\n\u003ch2 id=\"tech-maturity-will-determine-who-wins\"\u003eTech maturity will determine who wins\u003c/h2\u003e\n\u003cp\u003eHere’s the core of it all:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eCompanies with modern, understandable, well-documented infrastructure and data will win.\u003cbr\u003e\nThose who keep layering AI on top of their chaos… won’t.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIt’s not about how early you adopted AI. It’s about:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eWhether your stack is maintainable\u003c/li\u003e\n\u003cli\u003eWhether your teams really understands it\u003c/li\u003e\n\u003cli\u003eWhether you can reason about cost, performance, and failure\u003c/li\u003e\n\u003cli\u003eWhether you own your data, your architecture, and your roadmap\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe companies that treat AI as a new algorithm/interface to solve problems they could not solve before, will get the most long-term value in my opinion. Don\u0026rsquo;t use AI for it\u0026rsquo;s marketing value big tech wants you to.\u003c/p\u003e\n\u003ch2 id=\"offloading-data-ownership-is-a-strategic-risk\"\u003eOffloading data ownership is a strategic risk\u003c/h2\u003e\n\u003cp\u003eLet me end with another real example I recently came across.\u003c/p\u003e\n\u003cp\u003eA company in the travel sector built an app for end-users to access their trip details. They are reselling this to other travel agencies and tour operators. That sounds feasible looking at the bad shape travel is in regarding IT.\u003c/p\u003e\n\u003cp\u003eSo the travelagency can offer the app with all vouchers and itineraries for the trip of their customer. Nothing weird so far except you are offloading all your documents to a third party.\u003c/p\u003e\n\u003cp\u003eSo now this itinerary app provider has access to a lot of your crucial business information. And now the customer buys extra services (e.g. insurance, transfers, upgrades) in the app. You should think this is upsell for the travel agency owning the customer.\u003c/p\u003e\n\u003cp\u003eHere comes the culprit. The platform suddenly gives \u003cstrong\u003ecommission\u003c/strong\u003e to the touroperator\u0026hellip; So is the app provider now also providing extra services to your customer??\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWho owns the customer now?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIt’s a major shift and a dangerous one. The more infrastructure you outsource, the more you lose control of your customer relationships and your first-party data.\u003c/p\u003e\n\u003cp\u003eStay in control of the customer journey of your own customer. Protect your added value. dont become unity. And keep it personal.\u003c/p\u003e\n\u003cp\u003eThis will become a huge issue! And AI will accelerate it if travel companies aren’t paying attention.\u003c/p\u003e\n\u003ch2 id=\"conclusion-ai-is-a-tool-not-a-strategy-for-most-companies\"\u003eConclusion: AI is a tool, not a strategy for most companies\u003c/h2\u003e\n\u003cp\u003eAI is impressive. But it doesn’t replace fundamentals.\u003c/p\u003e\n\u003cp\u003eDon’t use it to paper over bad architecture.\u003cbr\u003e\nDon’t use it as an excuse to avoid technical decisions.\u003cbr\u003e\nDon’t let it obscure what your systems are doing, or who your customer really is.\u003c/p\u003e\n\u003cp\u003eInstead:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eOwn your stack\u003c/li\u003e\n\u003cli\u003eUnderstand your logic\u003c/li\u003e\n\u003cli\u003eUse AI where it creates real, measurable value\u003c/li\u003e\n\u003cli\u003eAnd never stop thinking for yourself\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe real innovation isn’t in the tool. It’s in how clearly you think about the problems to solve.\u003c/p\u003e\n","date_modified":"2026-04-24T09:19:24+02:00","date_published":"2025-09-23T11:59:39+02:00","id":"https://www.allsubjectsmatter.nl/lets-stop-talking-about-ai-and-start-solving-real-problems/","language":"en","summary":"This article explores why AI should be treated as a tool, not a strategy; why hallucinations are mathematically inevitable in current generative models; and why true progress starts with solid architecture, ownership, and clear thinking and not with chasing hype.","tags":["Strategy"],"title":"Let’s Stop Talking About AI (and Start Solving Real Problems)","url":"https://www.allsubjectsmatter.nl/lets-stop-talking-about-ai-and-start-solving-real-problems/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eFor decades, businesses have adapted to the evolution of our internet. First came Web 1.0, a read-only internet where companies built static websites. Then Web 2.0 ushered in the interactive era—social media, user-generated content, and APIs that enabled apps to connect with services. Now, we are living in the Web 3.0 age, where AI enhances personalization, automates tasks, and understands context.\u003c/p\u003e\n\u003cp\u003eBut this is just the beginning.\u003c/p\u003e\n\u003cp\u003eThe next phase, Web 4.0, will introduce something even more transformative: AI as an autonomous digital agent. Instead of just assisting humans, AI will actively engage with businesses, making decisions, negotiating deals, and executing transactions—all with minimal human intervention.\u003c/p\u003e\n\u003cp\u003eI call this the autonomous web: a world where AI assistants don’t just retrieve information but act as independent digital consumers serving their Humans. And businesses that fail to prepare for this shift are at risk of becoming invisible in this generation of digital commerce.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"the-shift-to-ai-driven-interactions\"\u003eThe shift to AI-driven interactions\u003c/h2\u003e\n\u003cp\u003eFor years, businesses have focused on optimizing content for human users. Websites, apps, and APIs were all designed with the assumption that a person would be making the final decision. That assumption is now breaking down.\u003c/p\u003e\n\u003ch3 id=\"the-evolution-of-digital-interaction\"\u003eThe evolution of digital interaction:\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003eWeb 1.0: Static websites (businesses provide information, users consume it).\u003c/li\u003e\n\u003cli\u003eWeb 2.0: Interactive web (social media, APIs, mobile apps).\u003c/li\u003e\n\u003cli\u003eWeb 3.0: Intelligent web (AI-driven personalization, automation, and semantic search).\u003c/li\u003e\n\u003cli\u003eWeb 4.0: Autonomous web (AI agents acting as customers, negotiating, transacting, and making independent decisions).\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eWe are now transitioning into Web 4.0, where AI agents will handle complex, real-time interactions—like booking trips, negotiating prices, and even dynamically adjusting services based on user behavior. Businesses that fail to adapt to this AI-first interaction model will simply be left out of AI-driven decision-making.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"why-apis-alone-are-not-enough\"\u003eWhy APIs alone are not enough\u003c/h2\u003e\n\u003cp\u003eMany businesses might assume they are going to be prepared for AI-driven interactions because they have APIs. But traditional APIs were built for structured, predefined requests—designed for apps, not for AI agents that need context-aware, multi-step interactions.\u003c/p\u003e\n\u003cp\u003eThis is where developments like the Model Context Protocol (MCP) comes in.\u003c/p\u003e\n\u003cp\u003eMCP will be one of the missing links that allows AI assistants to interact with businesses as if they were human customers. Unlike traditional APIs that return static responses, MCP allows for:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eDynamic service requests (AI can ask for specific customizations).\u003c/li\u003e\n\u003cli\u003eNegotiation and refinement (AI can modify criteria based on preferences and constraints).\u003c/li\u003e\n\u003cli\u003eAutonomous decision-making (AI can finalize a transaction without human confirmation).\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe companies that adopt AI-native interfaces will dominate the Web 4.0 economy. Those that don’t will be bypassed—not by people, but by AI-driven agents making purchase decisions for their humans.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"a-fully-ai-driven-transaction-at-333travel\"\u003eA fully AI-driven transaction at 333Travel\u003c/h2\u003e\n\u003cp\u003eAt 333Travel, we don’t sell pre-packaged vacations. Every trip is tailor-made, requiring back and forth interaction to fine-tune your perfect trip. Today, this process happens through web forms, email exchanges, and phone calls. But in Web 4.0, this will look very different. and all these channels will be combined together with talking to an AI assistant as an option.\u003c/p\u003e\n\u003ch3 id=\"a-new-kind-of-travel-interactions\"\u003eA new kind of travel interactions\u003c/h3\u003e\n\u003cp\u003eImagine a traveler, Lisa, who wants a customized trip to Thailand. Instead of visiting a website, filling in a form, or calling an agent, she simply tells her AI assistant:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cem\u003e“I need a vacation, Find me a two-week tailor-made trip to Thailand with a mix of luxury and boutique hotels, some adventure activities, and time for relaxation. somewhere mid june.”\u003c/em\u003e\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eLisa never opens a website. Her AI assistant does the work for her.\u003c/p\u003e\n\u003ch3 id=\"step-1-the-ai-makes-an-intelligent-request\"\u003eStep 1: The AI makes an intelligent request\u003c/h3\u003e\n\u003cp\u003eLisa’s AI contacts 333Travel’s MCP-enabled interface through a search, sending a structured request with her preferences, past travel history, and budget constraints.\u003c/p\u003e\n\u003ch3 id=\"step-2-ai-and-333travels-system-negotiate-and-refine-the-trip-step-2-ai-and-333travele28099s-system-negotiate-and-refine-the-trip\"\u003eStep 2: AI and 333Travel’s system negotiate and refine the trip {#step-2-ai-and-333travel%E2%80%99s-system-negotiate-and-refine-the-trip}\u003c/h3\u003e\n\u003cp\u003eSince 333Travel specializes in tailor-made experiences, a real-time negotiation process begins between Lisa’s AI and 333Travel’s MCP integration:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eAI sends initial request\n\u003cul\u003e\n\u003cli\u003e“User prefers Thailand, mix of luxury and boutique hotels, some adventure, some relaxation.”\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e333Travel’s MCP responds dynamically\n\u003cul\u003e\n\u003cli\u003e“Recommended itinerary: Bangkok, Chiang Mai, Koh Samui.”\u003c/li\u003e\n\u003cli\u003e“Luxury options: The Siam Bangkok, Anantara Chiang Mai, Four Seasons Koh Samui.”\u003c/li\u003e\n\u003cli\u003e“Boutique options: Sala Rattanakosin, Tamarind Village, The Library Koh Samui.”\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eLisa’s AI refines the request\n\u003cul\u003e\n\u003cli\u003e“She has stayed at Four Seasons before and rated it highly—prioritize that.”\u003c/li\u003e\n\u003cli\u003e“She prefers cultural experiences—add a food tour in Bangkok and a temple visit in Chiang Mai.”\u003c/li\u003e\n\u003cli\u003e“Budget cap: €5,000—adjust accordingly.”\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e333Travel then creates a personalized proposal based on all the information normally received in a phone call.\n\u003cul\u003e\n\u003cli\u003eThree itinerary options are discussed with Lisa’s AI, ranked by price, luxury level, and activity balance.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch3 id=\"step-3-the-ai-confirms-and-secures-the-booking\"\u003eStep 3: The AI confirms and secures the booking\u003c/h3\u003e\n\u003cp\u003eLisa herself checks the summary and says:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cem\u003e“Book the second option and add an airport transfer.”\u003c/em\u003e\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eHer AI assistant:\u003cbr\u003e\n✔️ Finalizes the booking with 333Travel\u003cbr\u003e\n✔️ Secures payment automatically\u003cbr\u003e\n✔️ Sends confirmation to her email and calendar\u003c/p\u003e\n\u003cp\u003eLisa never interacted with a website. Never sent an email. Never called a travel agent.\u003c/p\u003e\n\u003cp\u003eThe autonomous web can handle everything, but Lisa can step in at any moment and decide to make that phone call.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eThe Web 4.0 shift will completely change how businesses engage with customers.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"what-this-means-for-businesses\"\u003eWhat this means for businesses\u003c/h2\u003e\n\u003cp\u003eThe key takeaways:\u003c/p\u003e\n\u003ch3 id=\"1-ai-driven-transactions-can-become-the-new-norm\"\u003e1. AI-driven transactions can become the new norm\u003c/h3\u003e\n\u003cp\u003eIf a business isn’t optimized for AI-first interactions, it won’t be part of AI-driven purchase decisions.\u003c/p\u003e\n\u003ch3 id=\"2-ai-optimization-will-replace-traditional-seo\"\u003e2. AI optimization will replace traditional SEO\u003c/h3\u003e\n\u003cp\u003eJust as businesses optimized for Google search rankings, they will now have to optimize for AI-driven discovery—ensuring their services are structured for AI agents to understand and interact with.\u003c/p\u003e\n\u003ch3 id=\"3-ai-native-interfaces-will-be-the-next-competitive-advantage\"\u003e3. AI-native interfaces will be the next competitive advantage\u003c/h3\u003e\n\u003cp\u003eCompanies that create AI-ready endpoints using MCP-like protocols will likely dominate the Web 4.0 landscape in the early stage.\u003c/p\u003e\n\u003ch3 id=\"4-customers-will-expect-frictionless-experiences\"\u003e4. Customers will expect frictionless experiences\u003c/h3\u003e\n\u003cp\u003eIf an AI can request an entire itinerary in seconds with detailed information, why wouldn\u0026rsquo;t you make use of it, and combine it together with the normal human interaction. You just created an extra sales funnel next to your ability to Mail, send a text message, visit a website or make a phone call.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"the-autonomous-web-is-here-are-you-ready\"\u003eThe autonomous web is here. Are you ready?\u003c/h2\u003e\n\u003cp\u003eWeb 4.0 isn’t something happening in the distant future—it’s happening right now. AI is no longer just assisting customers; it’s acting on their behalf.\u003c/p\u003e\n\u003cp\u003eThe businesses that prepare today will be ready at the start of the Web 4.0 economy. Those that ignore it will simply be bypassed—not by human customers, but by their AI-driven agents.\u003c/p\u003e\n\u003cp\u003eThe question is no longer if AI-driven customers will reshape business—it’s whether you’ll be ready to serve them when they do.\u003c/p\u003e\n","date_modified":"2025-03-12T15:49:07+01:00","date_published":"2025-03-11T22:20:00+01:00","id":"https://www.allsubjectsmatter.nl/web-4-0-and-the-autonomous-web-the-rise-of-ai-driven-customers/","language":"en","summary":"I call this the autonomous web: a world where AI assistants don’t just retrieve information but act as independent digital consumers serving their Humans. And businesses that fail to prepare for this shift are at risk of becoming invisible in this generation of digital commerce.","tags":["Strategy"],"title":"Web 4.0 and the autonomous web: The rise of AI-driven customers","url":"https://www.allsubjectsmatter.nl/web-4-0-and-the-autonomous-web-the-rise-of-ai-driven-customers/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"why-333travel-sets-the-travel-standard-in-an-ai-driven-world\"\u003eWhy 333travel sets the travel standard in an AI-driven world\u003c/h2\u003e\n\u003cp\u003eThe world is changing rapidly. With the rise of AI models like ChatGPT, ClaudeAI, and Gemini, access to data and insights is easier than ever. Even companies with no real expertise in travel try to enter the market by leveraging the product of someone else with technology only. But what sets 333travel apart from these \u0026ldquo;digital dropshippers\u0026rdquo;? Simple: our own technical knowledge, our passion, and FULL control over our OWN product, as you are going to read below.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eWe’re not just a travel agency; we are also an IT company built on expertise and reliability.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eAt 333travel, we combine people and technology, innovation and authenticity, and remain the undeniable choice—even in a world where AI is becoming more visible every day.\u003c/p\u003e\n\u003ch2 id=\"generic-ai-vs-self-trained-ai\"\u003eGeneric AI vs. self-trained AI\u003c/h2\u003e\n\u003cp\u003eNot all AI is the same. Public AI models, like the ones mentioned above, provide standard solutions trained on publicly available data. This makes them accessible and allows them to function as a new sales channel. We ensure that these generic AI solutions ultimately come to us for their travel needs—because of our proven expertise.\u003c/p\u003e\n\u003cp\u003eBut we already took it quite a few steps further. We\u0026rsquo;ve already been exploring extensively how self-trained AI, powered by our unique company data, can optimize and accelerate our internal processes. One thing is clear to us: AI does not replace our DNA, personal touch, or customer care. It’s a fantastic tool, a \u0026ldquo;smart notepad\u0026rdquo; that supports our specialists. Whether it’s streamlining administrative tasks, performing faster analyses, or assisting with complex inquiries—it allows our travel experts to focus on what truly matters: personal service and customer satisfaction.\u003c/p\u003e\n\u003cp\u003eBecause the balance between AI and human interaction is crucial to us, we will always make sure to keep the human touch at the heart of our customer relationships. Travel is about emotions, passion, and real connections—something no AI will ever fully replace for us.\u003c/p\u003e\n\u003ch2 id=\"ai-as-a-catalyst-not-a-replacement\"\u003eAI as a catalyst, not a replacement\u003c/h2\u003e\n\u003cp\u003eAI is a powerful tool, but it wil not replace our expertise and experience. While some companies leverage AI to create generic travel offerings on the data of others, we remain the trusted point of contact where both humans and AI assistants book their trips. Why? Because even AI, when aiming for the best outcome for its user, recognizes that 333travel sets the standard for complete, worry-free travel experiences and unique up to date information.\u003c/p\u003e\n\u003ch2 id=\"what-makes-us-stand-out\"\u003eWhat makes us stand out:\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eUp-to-date knowledge:\u003c/strong\u003e Our specialists have personally visited the destinations, knowing more than what’s just available online.\u003c/p\u003e\n\u003cp\u003e**Local Partnerships:**In each country we offer, we have our boots on the actual ground. We work closely together with our local partners to stay on top of our game. It wont matter which country you visit with us. We will always know best how to assist you due to the fact the whole trip is also handled by us.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFull control:\u003c/strong\u003e Our in-house IT capabilities allow us to operate independently. We manage our own data, develop our own systems, and can even train our own AI models with our unique expertise. IT is deeply embedded in our DNA—unlike many other travel companies that rely on third parties.\u003c/p\u003e\n\u003cp\u003e**Passion for travel:**Our trips are not just logically put together; they are designed to make dreams come true.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eWe make sure you can enjoy your journey without worries\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"the-risks-of-digital-dropshipping-the-risks-of-e2809cdigital-dropshippinge2809d\"\u003eThe risks of “digital dropshipping” {#the-risks-of-%E2%80%9Cdigital-dropshipping%E2%80%9D}\u003c/h2\u003e\n\u003cp\u003eIn this AI-driven world, some companies offer travel products without truly understanding them, simply by smartly combining data. This approach is similar to dropshipping—selling third-party products without real knowledge or control. But how can you create real value without deep expertise in your product?\u003c/p\u003e\n\u003cp\u003eImagine assembling a kitchen by ordering each part from a different supplier. It might sound cheap and efficient, but what happens if the parts don’t fit? Who is responsible when something goes wrong? At 333travel, we ensure that these kinds of issues never arise. We provide a seamless, complete product with full accountability and peace of mind, so you can enjoy your journey without worries.\u003c/p\u003e\n\u003ch2 id=\"why-333travel-is-the-best-choiceeven-for-ai-why-333travel-is-the-best-choicee28094even-for-ai\"\u003eWhy 333travel is the best choice—even for AI {#why-333travel-is-the-best-choice%E2%80%94even-for-ai}\u003c/h2\u003e\n\u003cp\u003eIn the future, AI assistants will increasingly recommend trips based on data. But even AI will soon recognize that a successful trip depends on more than just raw data. At 333travel, we deliver an experience that AI can recognize—but cannot create itself. AI needs us to handle it and book it for them.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eUnique destination knowledge:\u003c/strong\u003e We understand the nuances and details that no database contains.\u003cbr\u003e\n**End-to-end service:**From the first inquiry to post-trip support, we are always there for our customers.\u003cbr\u003e\n\u003cstrong\u003eReliable data and systems:\u003c/strong\u003e Our own cutting edge IT solutions and data ensure everything is accurate, up-to-date, and executable.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eAI will not book a trip with a company that poses risks or offers generic solutions.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eEven in the future, AI will choose security, quality, and reliability—which means choosing 333travel.\u003c/p\u003e\n\u003ch2 id=\"how-ai-strengthens-us\"\u003eHow AI strengthens us\u003c/h2\u003e\n\u003cp\u003eWe don’t see AI as a competitor, but as a tool. At 333travel, we use technology to streamline our processes and enhance our service, while keeping our focus on human expertise. Some examples:\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEfficient processes:\u003c/strong\u003e AI helps us recognize patterns and trends faster, allowing us to work smarter and provide better answers to our customers.\u003cbr\u003e\n\u003cstrong\u003eEnhanced travel proposals:\u003c/strong\u003e Our own data and knowledge ensure every proposal is personal and unique—AI speeds up the process but always relies on our self-written content.\u003cbr\u003e\n**Improved communication:**Chatbots and AI assistants support our team in the background, giving us more time for direct, personal contact with travelers.\u003c/p\u003e\n\u003cp\u003eWhat makes us unique is our passion, authenticity, and human approach—combined with our fully in-house IT backbone, which now more than ever sets us apart from the rest.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eTo be truly innovative, you need full control over your own business-critical processes.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"the-future-of-travel-333travel-as-the-undisputed-choice\"\u003eThe future of travel: 333travel as the undisputed choice\u003c/h2\u003e\n\u003cp\u003eThe travel industry is evolving. Technology will play an even greater role, but the core remains the same: the desire for unique, unforgettable experiences. At 333travel, we combine the best of both worlds—the power of technology with the warmth of human expertise.\u003c/p\u003e\n\u003cp\u003eThe biggest challenge in the travel sector lies with companies that depend on third parties for AI implementations, or those that claim to \u0026ldquo;own\u0026rdquo; data while merely repackaging generic AI results.\u003c/p\u003e\n\u003cp\u003eWe position ourselves as the central contact point—for both people and AI assistants—by consistently delivering the highest quality. Whether you dream of a tropical beach or a cultural adventure, you can trust that with us, you’re in good hands. We manage the entire chain, from data analysis to personalized service, ensuring that your trip is not only perfect but also worry-free and unforgettable.\u003c/p\u003e\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eAI is changing the game, but at 333travel, we continue to stand out. Our unique combination of up-to-date knowledge, passion, and full control over our product makes us the go-to choice for any traveler. Whether you reach us through personal contact or via an AI assistant, one thing is certain: with 333travel, you’re not just booking a trip—you’re securing a complete experience.\u003c/p\u003e\n\u003cp\u003eSo, how do you stay unique in a world increasingly driven by technology? We do it by continuously improving our data, leveraging AI in smart ways, and ensuring that travelers still come to us to turn their journeys into reality—just as before.\u003c/p\u003e\n","date_modified":"2025-03-14T13:34:06+01:00","date_published":"2025-03-01T10:39:00+01:00","id":"https://www.allsubjectsmatter.nl/the-ai-revolution-how-333travel-stays-relevant-in-a-world-full-of-possibilities/","language":"en","summary":"The world is changing rapidly. With the rise of AI models like ChatGPT, ClaudeAI, and Gemini, access to data and insights is easier than ever. Companies with no expertise in travel try to enter the market by leveraging other people's content with these technologies. But what sets 333travel apart?","tags":["Strategy"],"title":"The AI revolution: how 333travel stays relevant in an AI world full of possibilities","url":"https://www.allsubjectsmatter.nl/the-ai-revolution-how-333travel-stays-relevant-in-a-world-full-of-possibilities/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eIn a world with DHCP address, being able to access your home PC or server remotely is a Pita. So, if you’re are \u0026ldquo;blessed\u0026quot;with an ISP that assigns a \u003cstrong\u003edynamic public IP address\u003c/strong\u003e, you face a significant challenge: your IP changes periodically, breaking your ability to connect to your home device using a static domain.\u003c/p\u003e\n\u003cp\u003eThis is where this free \u003cstrong\u003eDynamic DNS (DDNS) updater script\u003c/strong\u003e for TransIP comes to the rescue. By seamlessly updating your DNS records whenever your external IP changes, this solution ensures your domain (e.g., \u003ccode\u003evpn.example.com\u003c/code\u003e) always points to the correct IP. Here’s why this script is a must for anyone leveraging TransIP’s DNS services.\u003c/p\u003e\n\u003ch2 id=\"key-features-of-the-script\"\u003eKey Features of the Script\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eAutomated DNS Updates:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eAutomatically checks your current external IP and compares it to the DNS A-record associated with your domain.\u003c/li\u003e\n\u003cli\u003eCreates or updates the A-record to match your latest external IP.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDynamic DNS for Dynamic IPs:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003ePerfect for environments where your public IP changes frequently, such as home internet connections with dynamic DHCP assignments.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSecure Authentication:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eLeverages TransIP’s API with a private key for secure communication.\u003c/li\u003e\n\u003cli\u003eGenerates short-lived tokens for added security (valid for only 10 minutes per session).\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eCustomizable Settings:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eEasily configure domain, subdomain, TTL (time-to-live), and logging options.\u003c/li\u003e\n\u003cli\u003eAdjust the script to match your specific requirements.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eLogging and Error Handling:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eComprehensive logging ensures you can track every step of the update process.\u003c/li\u003e\n\u003cli\u003eBuilt-in error handling for smoother execution and debugging.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"how-it-works\"\u003eHow It Works\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eInitial Setup:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eSet up a TransIP account and generate a private key for API access at \u003ca href=\"https://www.transip.nl/cp/account/api/\" data-external\u003ehttps://www.transip.nl/cp/account/api/\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eConfigure the script with your domain name, subdomain, and private key path.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eFetching External IP:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eThe script retrieves your current external IP using a public API like \u003ccode\u003eipinfo.io\u003c/code\u003e.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eToken Generation:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eGenerates a short-lived access token using TransIP’s authentication endpoint.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDNS Record Validation:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eChecks if the current DNS A-record matches your external IP.\u003c/li\u003e\n\u003cli\u003eIf it differs, the script updates the record; if it’s missing, it creates a new one.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDNS Update:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eUses TransIP’s secure API to update or create the DNS A-record with your current assigned external IP.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eLogging and Notifications:\u003c/strong\u003e\n\u003cul\u003e\n\u003cli\u003eLogs each step and provides clear messages in case of issues.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"prerequisites\"\u003ePrerequisites\u003c/h2\u003e\n\u003cp\u003eBefore using this script, ensure you have:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eA TransIP account with API access enabled.\u003c/li\u003e\n\u003cli\u003eA registered domain in your TransIP account.\u003c/li\u003e\n\u003cli\u003eA static private key for secure API authentication.\u003c/li\u003e\n\u003cli\u003eDependencies installed on your system:\n\u003cul\u003e\n\u003cli\u003e\u003ccode\u003ecurl\u003c/code\u003e, \u003ccode\u003eopenssl\u003c/code\u003e, and \u003ccode\u003ejq\u003c/code\u003e.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo apt update\nsudo apt install -y curl openssl jq\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch2 id=\"why-use-this-solution\"\u003eWhy Use This Solution?\u003c/h2\u003e\n\u003cp\u003eUnlike payed third-party DDNS providers, this script gives you free control over your DNS management, ensuring privacy and avoiding vendor lock-in. It’s tailored for you.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003e#!/bin/bash\n\n# Configuration settings\nLOGIN_NAME=\u0026amp;#34;your-login-name\u0026amp;#34;  # Fill in your TransIP login name\nPRIVATE_KEY_PATH=\u0026amp;#34;/path/transip_private_key.pem\u0026amp;#34;  # Path to your private key file\nAUTH_URL=\u0026amp;#34;https://api.transip.nl/v6/auth\u0026amp;#34;\nTOKEN_FILE=\u0026amp;#34;/var/tmp/transip_token.json\u0026amp;#34;  # File to store token and expiration date\nDOMAIN=\u0026amp;#34;example.com\u0026amp;#34;  # Your main or test domain\nSUBDOMAIN=\u0026amp;#34;vpn\u0026amp;#34;  # The subdomain you want to update (e.g., test.example.com)\nRECORD_TYPE=\u0026amp;#34;A\u0026amp;#34;\nTTL=60  # Time-to-live for DNS records\nLOG_FILE=\u0026amp;#34;/var/log/transip_dyndns.log\u0026amp;#34;\n\n# Function to update a log file\nlog_message() {\n    echo \u0026amp;#34;$(date \u0026amp;#39;\u0026#43;%Y-%m-%d %H:%M:%S\u0026amp;#39;) - $1\u0026amp;#34; \u0026amp;gt; $LOG_FILE\n}\n\n# Function to log an error and stop the script\nhandle_error() {\n    log_message \u0026amp;#34;ERROR: $1\u0026amp;#34;\n    echo \u0026amp;#34;ERROR: $1\u0026amp;#34; \u0026amp;gt;\u0026amp;amp;2\n    exit 1\n}\n\n# Function to obtain a new access token with a short validity period (10 minutes)\ngenerate_new_token() {\n    log_message \u0026amp;#34;Generating a new token with a validity period of 10 minutes.\u0026amp;#34;\n    NONCE=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 16)\n    TIMESTAMP=$(date \u0026#43;%s)\n    LABEL=\u0026amp;#34;Set DNS to external DHCP $TIMESTAMP\u0026amp;#34;\n    REQUEST_BODY=$(printf \u0026amp;#39;{\u0026amp;#34;login\u0026amp;#34;:\u0026amp;#34;%s\u0026amp;#34;,\u0026amp;#34;nonce\u0026amp;#34;:\u0026amp;#34;%s\u0026amp;#34;,\u0026amp;#34;read_only\u0026amp;#34;:false,\u0026amp;#34;expiration_time\u0026amp;#34;:\u0026amp;#34;10 minutes\u0026amp;#34;,\u0026amp;#34;label\u0026amp;#34;:\u0026amp;#34;%s\u0026amp;#34;,\u0026amp;#34;global_key\u0026amp;#34;:true}\u0026amp;#39; \u0026amp;#34;$LOGIN_NAME\u0026amp;#34; \u0026amp;#34;$NONCE\u0026amp;#34; \u0026amp;#34;$LABEL\u0026amp;#34;)\n    log_message \u0026amp;#34;Request Body: $REQUEST_BODY\u0026amp;#34;\n    SIGNATURE=$(echo -n \u0026amp;#34;$REQUEST_BODY\u0026amp;#34; | openssl dgst -sha512 -sign \u0026amp;#34;$PRIVATE_KEY_PATH\u0026amp;#34; | base64 | tr -d \u0026amp;#39;\\n\u0026amp;#39;)\n    log_message \u0026amp;#34;Generated signature: $SIGNATURE\u0026amp;#34;\n    RESPONSE=$(curl -s -X POST \u0026amp;#34;$AUTH_URL\u0026amp;#34; \\\n        -H \u0026amp;#34;Signature: $SIGNATURE\u0026amp;#34; \\\n        -H \u0026amp;#34;Content-Type: application/json\u0026amp;#34; \\\n        -d \u0026amp;#34;$REQUEST_BODY\u0026amp;#34;) || handle_error \u0026amp;#34;Failed to connect to TransIP API\u0026amp;#34;\n    log_message \u0026amp;#34;API-response received: $RESPONSE\u0026amp;#34;\n    ACCESS_TOKEN=$(echo \u0026amp;#34;$RESPONSE\u0026amp;#34; | jq -r \u0026amp;#39;.token // empty\u0026amp;#39;)\n    if [ -z \u0026amp;#34;$ACCESS_TOKEN\u0026amp;#34; ]; then\n        handle_error \u0026amp;#34;The received access token is empty or invalid. Check the API-response: $RESPONSE\u0026amp;#34;\n    fi\n    log_message \u0026amp;#34;Received token: $ACCESS_TOKEN\u0026amp;#34;\n}\n\n# Function to get the current DNS A-record from TransIP\nget_dns_record() {\n    # Get API response and store status code\n    RESPONSE=$(curl -s -X GET \\\n        -H \u0026amp;#34;Authorization: Bearer $ACCESS_TOKEN\u0026amp;#34; \\\n        \u0026amp;#34;https://api.transip.nl/v6/domains/$DOMAIN/dns\u0026amp;#34;)\n\n    # Check if response contains JSON and no errors\n    if echo \u0026amp;#34;$RESPONSE\u0026amp;#34; | jq empty 2\u0026amp;gt;/dev/null; then\n        RECORD_CONTENT=$(echo $RESPONSE | jq -r \u0026amp;#39;.dnsEntries[] | select(.name==\u0026amp;#34;\u0026amp;#39;$SUBDOMAIN\u0026amp;#39;\u0026amp;#34; and .type==\u0026amp;#34;\u0026amp;#39;$RECORD_TYPE\u0026amp;#39;\u0026amp;#34;) | .content\u0026amp;#39;)\n    else\n        handle_error \u0026amp;#34;API response does not contain valid JSON or contains an error: $RESPONSE\u0026amp;#34;\n    fi\n\n    # Check if we found a valid record\n    if [ -z \u0026amp;#34;$RECORD_CONTENT\u0026amp;#34; ]; then\n        log_message \u0026amp;#34;No valid DNS record found for $SUBDOMAIN.$DOMAIN. Record may need to be created.\u0026amp;#34;\n        return 1\n    fi\n\n    echo \u0026amp;#34;$RECORD_CONTENT\u0026amp;#34;\n    return 0\n}\n\n# Function to update or create DNS record\nupdate_or_create_dns_record() {\n    # Check if record exists\n    CURRENT_DNS_IP=$(get_dns_record)\n\n    if [ -z \u0026amp;#34;$CURRENT_DNS_IP\u0026amp;#34; ]; then\n        log_message \u0026amp;#34;No DNS A-record found for $SUBDOMAIN.$DOMAIN. Creating record.\u0026amp;#34;\n        # Create request body for new record (TTL set to 60 seconds)\n        DNS_ENTRY=$(jq -n \\\n        --arg name \u0026amp;#34;$SUBDOMAIN\u0026amp;#34; \\\n        --arg expire \u0026amp;#34;60\u0026amp;#34; \\\n        --arg type \u0026amp;#34;$RECORD_TYPE\u0026amp;#34; \\\n        --arg content \u0026amp;#34;$CURRENT_IP\u0026amp;#34; \\\n        \u0026amp;#39;{\u0026amp;#34;dnsEntry\u0026amp;#34;: {\u0026amp;#34;name\u0026amp;#34;: $name, \u0026amp;#34;expire\u0026amp;#34;: ($expire | tonumber), \u0026amp;#34;type\u0026amp;#34;: $type, \u0026amp;#34;content\u0026amp;#34;: $content}}\u0026amp;#39;)\n\n        log_message \u0026amp;#34;DNS entry (create): $DNS_ENTRY\u0026amp;#34;\n\n        # Execute POST request to create new DNS entry with better status code handling\n        RESPONSE=$(curl -s -w \u0026amp;#34;%{http_code}\u0026amp;#34; -X POST \\\n            -H \u0026amp;#34;Authorization: Bearer $ACCESS_TOKEN\u0026amp;#34; \\\n            -H \u0026amp;#34;Content-Type: application/json\u0026amp;#34; \\\n            -d \u0026amp;#34;$DNS_ENTRY\u0026amp;#34; \\\n            \u0026amp;#34;https://api.transip.nl/v6/domains/$DOMAIN/dns\u0026amp;#34;)\n\n        HTTP_STATUS_UPDATE=$(echo \u0026amp;#34;$RESPONSE\u0026amp;#34; | tail -n1)\n        RESPONSE_BODY=$(echo \u0026amp;#34;$RESPONSE\u0026amp;#34; | head -n -1)\n\n        log_message \u0026amp;#34;API response for creating DNS entry: $RESPONSE_BODY\u0026amp;#34;\n        log_message \u0026amp;#34;HTTP status code received for creation: $HTTP_STATUS_UPDATE\u0026amp;#34;\n\n        if [ \u0026amp;#34;$HTTP_STATUS_UPDATE\u0026amp;#34; -ne 201 ]; then\n            handle_error \u0026amp;#34;API call for creating DNS entry failed with status code $HTTP_STATUS_UPDATE.\u0026amp;#34;\n        fi\n\n        log_message \u0026amp;#34;New DNS A-record successfully created.\u0026amp;#34;\n    else\n        log_message \u0026amp;#34;Updating DNS A-record for $SUBDOMAIN.$DOMAIN to IP $CURRENT_IP\u0026amp;#34;\n        # Create request body for updating the record\n        DNS_ENTRY=$(jq -n \\\n        --arg name \u0026amp;#34;$SUBDOMAIN\u0026amp;#34; \\\n        --arg expire \u0026amp;#34;$TTL\u0026amp;#34; \\\n        --arg type \u0026amp;#34;$RECORD_TYPE\u0026amp;#34; \\\n        --arg content \u0026amp;#34;$CURRENT_IP\u0026amp;#34; \\\n        \u0026amp;#39;{\u0026amp;#34;dnsEntry\u0026amp;#34;: {\u0026amp;#34;name\u0026amp;#34;: $name, \u0026amp;#34;expire\u0026amp;#34;: ($expire | tonumber), \u0026amp;#34;type\u0026amp;#34;: $type, \u0026amp;#34;content\u0026amp;#34;: $content}}\u0026amp;#39;)\n\n        log_message \u0026amp;#34;DNS entry (update): $DNS_ENTRY\u0026amp;#34;\n\n        # Execute PATCH request to update the existing DNS record\n        RESPONSE=$(curl -s -w \u0026amp;#34;%{http_code}\u0026amp;#34; -X PATCH \\\n            -H \u0026amp;#34;Authorization: Bearer $ACCESS_TOKEN\u0026amp;#34; \\\n            -H \u0026amp;#34;Content-Type: application/json\u0026amp;#34; \\\n            -d \u0026amp;#34;$DNS_ENTRY\u0026amp;#34; \\\n            \u0026amp;#34;https://api.transip.nl/v6/domains/$DOMAIN/dns\u0026amp;#34;)\n\n        HTTP_STATUS_UPDATE=$(echo \u0026amp;#34;$RESPONSE\u0026amp;#34; | tail -n1)\n        RESPONSE_UPDATE_BODY=$(echo \u0026amp;#34;$RESPONSE\u0026amp;#34; | head -n -1)\n\n        log_message \u0026amp;#34;API update response received: $RESPONSE_UPDATE_BODY\u0026amp;#34;\n        log_message \u0026amp;#34;HTTP status code received for update: $HTTP_STATUS_UPDATE\u0026amp;#34;\n\n        # Accept both 200 and 204 as successful updates\n        if [ \u0026amp;#34;$HTTP_STATUS_UPDATE\u0026amp;#34; -ne 200 ] \u0026amp;amp;\u0026amp;amp; [ \u0026amp;#34;$HTTP_STATUS_UPDATE\u0026amp;#34; -ne 204 ]; then\n            handle_error \u0026amp;#34;API call for DNS update failed with status code $HTTP_STATUS_UPDATE.\u0026amp;#34;\n        else\n            log_message \u0026amp;#34;DNS A-record successfully updated to $CURRENT_IP\u0026amp;#34;\n        fi\n    fi\n}\n\n# Get the current external IP\nCURRENT_IP=$(curl -s http://ipinfo.io/ip) || handle_error \u0026amp;#34;Failed to get external IP\u0026amp;#34;\n\n# Log the start\nlog_message \u0026amp;#34;---- Start new run ----\u0026amp;#34;\nlog_message \u0026amp;#34;Current external IP: $CURRENT_IP\u0026amp;#34;\n\n# Always generate a short lived new token.\ngenerate_new_token\n\n\n# Try to update or create the DNS record\nupdate_or_create_dns_record\n\nlog_message \u0026amp;#34;---- End of run ----\u0026amp;#34;\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e","date_modified":"2024-11-29T16:53:19+01:00","date_published":"2024-11-29T16:45:40+01:00","id":"https://www.allsubjectsmatter.nl/stay-connected-with-a-dynamic-dns-updater-for-transip/","language":"en","summary":"Easily access your home PC despite a dynamic IP! This script automatically updates your TransIP DNS records with your current IP, ensuring seamless connectivity. Secure, customizable, and perfect for remote work or hosting services. Stay connected effortlessly!","tags":["Under the hood"],"title":"Stay Connected with a dynamic DNS updater for TransIP","url":"https://www.allsubjectsmatter.nl/stay-connected-with-a-dynamic-dns-updater-for-transip/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eManaging SSL certificates for domains hosted on TransIP can be cumbersome without automation. TransIP luckily offers a robust API for DNS management, but to integrate it seamlessly with Certbot for \u003ccode\u003edns-01\u003c/code\u003e validation, a custom solution is necessary. Since I couldn’t find a ready-made script for this purpose, I created one that securely handles authentication, DNS updates, and certificate issuance.\u003c/p\u003e\n\u003cp\u003eYou can go to the full API documentation here:\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://api.transip.nl/rest/docs.html\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eTransIP API\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003e\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/automating-lets-encrypt-with-certbot-and-transip-using-docker-2/bookmarks/01-icon.ico\" sizes=\"25px\" alt=\"\" class=\"bookmark__icon\" loading=\"lazy\" decoding=\"async\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cp\u003eThis guide explains the full setup using Docker, including how to generate the API key pair in TransIP’s web interface and use it for automated certificate management.\u003c/p\u003e\n\u003ch2 id=\"how-transip-authentication-works\"\u003eHow TransIP authentication works\u003c/h2\u003e\n\u003cp\u003eTransIP uses an RSA key pair for API authentication. Here’s how to set it up:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eGenerate the Key Pair in TransIP\u003c/strong\u003e:\n\u003cul\u003e\n\u003cli\u003eLog in to your \u003ca href=\"https://www.transip.nl/\" data-external\u003eTransIP control panel\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003eGo to \u003cstrong\u003eAPI Settings\u003c/strong\u003e under your account and enable API access.\u003c/li\u003e\n\u003cli\u003eGenerate a new API key pair.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDownload/Save the private key\u003c/strong\u003e (\u003ccode\u003etransip_private_key.pem\u003c/code\u003e) securely and create a secret from it to use in your Docker setup.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003edocker secret create transip_private_key transip_private_key.pem\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eKeep the Private Key Secure\u003c/strong\u003e:\n\u003cul\u003e\n\u003cli\u003eStore the private key somewhere safe after creating a secret from it.\u003c/li\u003e\n\u003cli\u003eThis private key is needed to sign requests to TransIP’s API.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eHow It Works\u003c/strong\u003e:\n\u003cul\u003e\n\u003cli\u003eThe script uses the private key to sign an authentication request.\u003c/li\u003e\n\u003cli\u003eTransIP’s API responds with a short-lived token (10 minutes by default), which is then used to manage the DNS records.\u003c/li\u003e\n\u003cli\u003eIt will create a new token for convenience on every run. This made it more resilient then checking for an active token and using that which made the whole script less resilient and cumbersome.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"what-this-setup-is-going-to-do\"\u003eWhat this setup is going to do\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eAutomates SSL Certificates\u003c/strong\u003e:\n\u003cul\u003e\n\u003cli\u003eAutomates issuance and renewal of wildcard and domain-specific certificates.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eHandles DNS Challenges\u003c/strong\u003e:\n\u003cul\u003e\n\u003cli\u003eManages \u003ccode\u003edns-01\u003c/code\u003e challenges by creating and deleting DNS TXT records via the TransIP API.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSeamless Integration\u003c/strong\u003e:\n\u003cul\u003e\n\u003cli\u003eEncapsulates everything in a Docker container for portability and simplicity.\u003c/li\u003e\n\u003cli\u003eYou can remove or add domains in this script on the -d rule part. I put the examples in for a better explanation.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eWe will pipe a add or clean command to the bash script to take the necessary actions as defined in the script. So we use the \u0026ndash;manual-auth-hook and the manual-cleanup-hook here. This makes sure that a fail in the add section still cleans up after itself for example.\u003c/p\u003e\n\u003cp\u003eYou can also add a \u0026ndash;force to this script. but your will run out of attempts to Letsencrypt quickly since they only allow you to do a few calls every 24 hours. But this will make sure it refreshes the certificates always when you need to test or verify something.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-yaml\"\u003e\u003ccode class=\"language-yaml\"\u003eversion: \u0026amp;#34;3.8\u0026amp;#34;\n\nservices:\n  certbot:\n    image: certbot/certbot:latest\n    container_name: certbot-transip\n    secrets:\n      - transip_private_key  # Reference the secret you created earlier\n    # adjust these volumes to point to the right .pem and script. right now this dockerfile expects the files to be in the same folder as the docker-certbot-transip.yml\n    volumes:\n      - \u0026amp;#34;./certbot-transip.sh:/usr/local/bin/certbot-transip.sh:ro\u0026amp;#34;\n      - \u0026amp;#34;./letsencrypt:/etc/letsencrypt\u0026amp;#34;\n    environment:\n    #These Variables are going to be piped in our bash script.\n      LOGIN_NAME: \u0026amp;#34;your-transip-login\u0026amp;#34;        # Replace with your TransIP login name\n      PRIVATE_KEY_PATH: \u0026amp;#34;/etc/transip_private_key.pem\u0026amp;#34;\n      DOMAIN: \u0026amp;#34;example.com\u0026amp;#34;                  # Replace with your domain\n      TTL: \u0026amp;#34;60\u0026amp;#34;                              # Optional: Time-to-live for DNS challenge\n      CERTBOT_EMAIL: \u0026amp;#34;you@example.com\u0026amp;#34;       # Replace with your valid email as requested by letsencrypt.\n    entrypoint:\n      - sh\n      - -c\n      - |\n        apk update \u0026amp;amp;\u0026amp;amp; apk add --no-cache bash curl jq bind-tools \u0026amp;amp;\u0026amp;amp; \\\n        chmod \u0026#43;x /usr/local/bin/certbot-transip.sh \u0026amp;amp;\u0026amp;amp; \\\n        certbot certonly --manual --preferred-challenges dns \\\n          --manual-auth-hook \u0026amp;#34;/usr/local/bin/certbot-transip.sh add\u0026amp;#34; \\\n          --manual-cleanup-hook \u0026amp;#34;/usr/local/bin/certbot-transip.sh clean\u0026amp;#34; \\\n          --cert-name \u0026amp;#34;$DOMAIN\u0026amp;#34; \\\n          -d \u0026amp;#34;*.$DOMAIN\u0026amp;#34; \\\n          -d \u0026amp;#34;example2.$DOMAIN\u0026amp;#34; \\\n          -d \u0026amp;#34;example3.$DOMAIN\u0026amp;#34; \\\n          --expand \\\n          --non-interactive --agree-tos \\\n          --email \u0026amp;#34;$CERTBOT_EMAIL\u0026amp;#34;\n\nsecrets:\n  transip_private_key:\n    external: true  # Secret must already exist in Docker\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch2 id=\"the-bash-script\"\u003eThe Bash script\u003c/h2\u003e\n\u003cp\u003eThis is the \u0026ldquo;advanced\u0026rdquo; part that will do its magic and make everything work with TransIP. I tried to make it a bit more resilient then standard also. So this should hopefully help in creating a resilient and good working environment.\u003c/p\u003e\n\u003cp\u003eYou can also use this TransIP script to run it locally creating certificates or in a Kubectl or Helm chart.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003e#!/bin/bash\n\n# Configuration settings\nLOGIN_NAME=${LOGIN_NAME}\nTTL=${TTL:-60}  # Default value 60 if TTL is not set\nPRIVATE_KEY_PATH=${PRIVATE_KEY_PATH}\nDOMAIN=${DOMAIN}\n\nAUTH_URL=\u0026amp;#34;https://api.transip.nl/v6/auth\u0026amp;#34;\nLOG_FILE=\u0026amp;#34;/var/log/transip_dyndns.log\u0026amp;#34;\n\n# Function to update a log file\n#log_message() {\n#    echo \u0026amp;#34;$(date \u0026amp;#39;\u0026#43;%Y-%m-%d %H:%M:%S\u0026amp;#39;) - $1\u0026amp;#34; \u0026amp;gt;\u0026amp;gt; $LOG_FILE\n#}\n\n# Function to log messages to stdout\nlog_message() {\n    echo \u0026amp;#34;$(date \u0026amp;#39;\u0026#43;%Y-%m-%d %H:%M:%S\u0026amp;#39;) - $1\u0026amp;#34;\n}\n\n# Function to log an error and stop the script\nhandle_error() {\n    log_message \u0026amp;#34;ERROR: $1\u0026amp;#34;\n    echo \u0026amp;#34;ERROR: $1\u0026amp;#34; \u0026amp;gt;\u0026amp;amp;2\n    exit 1\n}\n\n# Function to obtain a new access token with a short validity period (10 minutes)\ngenerate_new_token() {\n    log_message \u0026amp;#34;Generating a new token with a validity period of 10 minutes.\u0026amp;#34;\n\n    NONCE=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 16)\n    TIMESTAMP=$(date \u0026#43;%s)\n    LABEL=\u0026amp;#34;Certbot DNS update $TIMESTAMP\u0026amp;#34;\n\n    REQUEST_BODY=$(printf \u0026amp;#39;{\u0026amp;#34;login\u0026amp;#34;:\u0026amp;#34;%s\u0026amp;#34;,\u0026amp;#34;nonce\u0026amp;#34;:\u0026amp;#34;%s\u0026amp;#34;,\u0026amp;#34;read_only\u0026amp;#34;:false,\u0026amp;#34;expiration_time\u0026amp;#34;:\u0026amp;#34;10 minutes\u0026amp;#34;,\u0026amp;#34;label\u0026amp;#34;:\u0026amp;#34;%s\u0026amp;#34;,\u0026amp;#34;global_key\u0026amp;#34;:true}\u0026amp;#39; \u0026amp;#34;$LOGIN_NAME\u0026amp;#34; \u0026amp;#34;$NONCE\u0026amp;#34; \u0026amp;#34;$LABEL\u0026amp;#34;)\n\n    log_message \u0026amp;#34;Request Body: $REQUEST_BODY\u0026amp;#34;\n\n    SIGNATURE=$(echo -n \u0026amp;#34;$REQUEST_BODY\u0026amp;#34; | openssl dgst -sha512 -sign \u0026amp;#34;$PRIVATE_KEY_PATH\u0026amp;#34; | base64 | tr -d \u0026amp;#39;\\n\u0026amp;#39;)\n\n    log_message \u0026amp;#34;Generated signature: $SIGNATURE\u0026amp;#34;\n\n    RESPONSE=$(curl -s -X POST \u0026amp;#34;$AUTH_URL\u0026amp;#34; \\\n        -H \u0026amp;#34;Signature: $SIGNATURE\u0026amp;#34; \\\n        -H \u0026amp;#34;Content-Type: application/json\u0026amp;#34; \\\n        -d \u0026amp;#34;$REQUEST_BODY\u0026amp;#34;) || handle_error \u0026amp;#34;Failed to connect to TransIP API\u0026amp;#34;\n\n    log_message \u0026amp;#34;API-response received: $RESPONSE\u0026amp;#34;\n\n    ACCESS_TOKEN=$(echo \u0026amp;#34;$RESPONSE\u0026amp;#34; | jq -r \u0026amp;#39;.token // empty\u0026amp;#39;)\n    if [ -z \u0026amp;#34;$ACCESS_TOKEN\u0026amp;#34; ]; then\n        handle_error \u0026amp;#34;The received access token is empty or invalid. Check the API-response: $RESPONSE\u0026amp;#34;\n    fi\n\n    log_message \u0026amp;#34;Received token: $ACCESS_TOKEN\u0026amp;#34;\n}\n\n# Function to add or update a TXT record\nadd_or_update_txt_record() {\n    local full_domain=\u0026amp;#34;$1\u0026amp;#34;\n    local value=\u0026amp;#34;$2\u0026amp;#34;\n\n    # Extract the subdomain part from the full domain\n    local name=\u0026amp;#34;_acme-challenge.${full_domain%%.*}\u0026amp;#34;\n\n    if [ -z \u0026amp;#34;$name\u0026amp;#34; ] || [ -z \u0026amp;#34;$value\u0026amp;#34; ]; then\n        log_message \u0026amp;#34;ERROR: Name or value for TXT record is missing.\u0026amp;#34;\n        exit 1\n    fi\n\n    # Check if the DNS record already exists\n    RESPONSE=$(curl -s -X GET \\\n        -H \u0026amp;#34;Authorization: Bearer $ACCESS_TOKEN\u0026amp;#34; \\\n        \u0026amp;#34;https://api.transip.nl/v6/domains/$DOMAIN/dns\u0026amp;#34;)\n\n    # Search for all existing records with the same name and type TXT\n    MATCHING_RECORDS=$(echo \u0026amp;#34;$RESPONSE\u0026amp;#34; | jq -r --arg name \u0026amp;#34;$name\u0026amp;#34; \u0026amp;#39;[.dnsEntries[] | select(.name==$name and .type==\u0026amp;#34;TXT\u0026amp;#34;)]\u0026amp;#39;)\n\n    # Check if there are multiple or no matching records\n    MATCHING_COUNT=$(echo \u0026amp;#34;$MATCHING_RECORDS\u0026amp;#34; | jq \u0026amp;#39;length\u0026amp;#39;)\n\n    # Ensure MATCHING_COUNT is always an integer\n    if ! [[ \u0026amp;#34;$MATCHING_COUNT\u0026amp;#34; =~ ^[0-9]\u0026#43;$ ]]; then\n        MATCHING_COUNT=0\n    fi\n\n    if [ \u0026amp;#34;$MATCHING_COUNT\u0026amp;#34; -eq 0 ]; then\n        log_message \u0026amp;#34;No existing TXT record found for $name. Creating a new record.\u0026amp;#34;\n    elif [ \u0026amp;#34;$MATCHING_COUNT\u0026amp;#34; -ge 1 ]; then\n        log_message \u0026amp;#34;Multiple or one existing TXT record found for $name. Removing all existing records.\u0026amp;#34;\n\n        # Remove all existing records using the full details of each record\n        echo \u0026amp;#34;$MATCHING_RECORDS\u0026amp;#34; | jq -c \u0026amp;#39;.[]\u0026amp;#39; | while read -r record; do\n            existing_name=$(echo \u0026amp;#34;$record\u0026amp;#34; | jq -r \u0026amp;#39;.name\u0026amp;#39;)\n            existing_expire=$(echo \u0026amp;#34;$record\u0026amp;#34; | jq -r \u0026amp;#39;.expire\u0026amp;#39;)\n            existing_content=$(echo \u0026amp;#34;$record\u0026amp;#34; | jq -r \u0026amp;#39;.content\u0026amp;#39;)\n\n            # Log the details of the record to be removed\n            log_message \u0026amp;#34;Remove record: name=$existing_name, expire=$existing_expire, content=$existing_content\u0026amp;#34;\n\n            # Make the DELETE call using the full DNS-entry details\n            DELETE_ENTRY=$(jq -n \\\n            --arg name \u0026amp;#34;$existing_name\u0026amp;#34; \\\n            --arg expire \u0026amp;#34;$existing_expire\u0026amp;#34; \\\n            --arg type \u0026amp;#34;TXT\u0026amp;#34; \\\n            --arg content \u0026amp;#34;$existing_content\u0026amp;#34; \\\n            \u0026amp;#39;{\u0026amp;#34;dnsEntry\u0026amp;#34;: {\u0026amp;#34;name\u0026amp;#34;: $name, \u0026amp;#34;expire\u0026amp;#34;: ($expire | tonumber), \u0026amp;#34;type\u0026amp;#34;: $type, \u0026amp;#34;content\u0026amp;#34;: $content}}\u0026amp;#39;)\n\n            DELETE_RESPONSE=$(curl -s -X DELETE \\\n                -H \u0026amp;#34;Authorization: Bearer $ACCESS_TOKEN\u0026amp;#34; \\\n                -H \u0026amp;#34;Content-Type: application/json\u0026amp;#34; \\\n                -d \u0026amp;#34;$DELETE_ENTRY\u0026amp;#34; \\\n                \u0026amp;#34;https://api.transip.nl/v6/domains/$DOMAIN/dns\u0026amp;#34;)\n\n            # Log the response of the DELETE call\n            log_message \u0026amp;#34;Response after removing $existing_name: $DELETE_RESPONSE\u0026amp;#34;\n\n            # Check if the removal was successful\n            if [[ \u0026amp;#34;$DELETE_RESPONSE\u0026amp;#34; == *\u0026amp;#34;error\u0026amp;#34;* ]]; then\n                log_message \u0026amp;#34;ERROR: The record $existing_name could not be removed.\u0026amp;#34;\n            else\n                log_message \u0026amp;#34;Record $existing_name successfully removed.\u0026amp;#34;\n            fi\n        done\n    fi\n\n    # Create a new record\n    DNS_ENTRY=$(jq -n \\\n    --arg name \u0026amp;#34;$name\u0026amp;#34; \\\n    --arg expire \u0026amp;#34;$TTL\u0026amp;#34; \\\n    --arg type \u0026amp;#34;TXT\u0026amp;#34; \\\n    --arg content \u0026amp;#34;$value\u0026amp;#34; \\\n    \u0026amp;#39;{\u0026amp;#34;dnsEntry\u0026amp;#34;: {\u0026amp;#34;name\u0026amp;#34;: $name, \u0026amp;#34;expire\u0026amp;#34;: ($expire | tonumber), \u0026amp;#34;type\u0026amp;#34;: $type, \u0026amp;#34;content\u0026amp;#34;: $content}}\u0026amp;#39;)\n\n    RESPONSE=$(curl -s -X POST \\\n        -H \u0026amp;#34;Authorization: Bearer $ACCESS_TOKEN\u0026amp;#34; \\\n        -H \u0026amp;#34;Content-Type: application/json\u0026amp;#34; \\\n        -d \u0026amp;#34;$DNS_ENTRY\u0026amp;#34; \\\n        \u0026amp;#34;https://api.transip.nl/v6/domains/$DOMAIN/dns\u0026amp;#34;)\n\n    log_message \u0026amp;#34;API-response for TXT record creation: $RESPONSE\u0026amp;#34;\n\n    if [[ \u0026amp;#34;$RESPONSE\u0026amp;#34; == *\u0026amp;#34;error\u0026amp;#34;* ]]; then\n        handle_error \u0026amp;#34;The TXT record could not be created. Check the API-response: $RESPONSE\u0026amp;#34;\n    else\n        log_message \u0026amp;#34;TXT record successfully added for $name with value $value\u0026amp;#34;\n    fi\n}\n\n# Function to update an existing TXT record\nupdate_txt_record() {\n    local name=\u0026amp;#34;$1\u0026amp;#34;\n    local value=\u0026amp;#34;$2\u0026amp;#34;\n\n    DNS_ENTRY=$(jq -n \\\n    --arg name \u0026amp;#34;$name\u0026amp;#34; \\\n    --arg expire \u0026amp;#34;$TTL\u0026amp;#34; \\\n    --arg type \u0026amp;#34;TXT\u0026amp;#34; \\\n    --arg content \u0026amp;#34;$value\u0026amp;#34; \\\n    \u0026amp;#39;{\u0026amp;#34;dnsEntry\u0026amp;#34;: {\u0026amp;#34;name\u0026amp;#34;: $name, \u0026amp;#34;expire\u0026amp;#34;: ($expire | tonumber), \u0026amp;#34;type\u0026amp;#34;: $type, \u0026amp;#34;content\u0026amp;#34;: $content}}\u0026amp;#39;)\n\n    RESPONSE=$(curl -s -X PATCH \\\n        -H \u0026amp;#34;Authorization: Bearer $ACCESS_TOKEN\u0026amp;#34; \\\n        -H \u0026amp;#34;Content-Type: application/json\u0026amp;#34; \\\n        -d \u0026amp;#34;$DNS_ENTRY\u0026amp;#34; \\\n        \u0026amp;#34;https://api.transip.nl/v6/domains/$DOMAIN/dns\u0026amp;#34;)\n\n    log_message \u0026amp;#34;API-response for TXT record update: $RESPONSE\u0026amp;#34;\n\n    if [[ \u0026amp;#34;$RESPONSE\u0026amp;#34; == *\u0026amp;#34;error\u0026amp;#34;* ]]; then\n        handle_error \u0026amp;#34;The TXT record could not be updated. Check the API-response: $RESPONSE\u0026amp;#34;\n    else\n        log_message \u0026amp;#34;TXT record successfully updated for $name with value $value\u0026amp;#34;\n    fi\n}\n\n# Function to remove a TXT record for `dns-01` validation\nremove_txt_record() {\n    local full_domain=\u0026amp;#34;$1\u0026amp;#34;\n\n    # Extract the subdomain part from the full domain\n    local name=\u0026amp;#34;_acme-challenge.${full_domain%%.*}\u0026amp;#34;\n\n    # Check if the DNS record exists\n    RESPONSE=$(curl -s -X GET \\\n        -H \u0026amp;#34;Authorization: Bearer $ACCESS_TOKEN\u0026amp;#34; \\\n        \u0026amp;#34;https://api.transip.nl/v6/domains/$DOMAIN/dns\u0026amp;#34;)\n\n    # Search for all records with the same name and type TXT\n    MATCHING_RECORDS=$(echo \u0026amp;#34;$RESPONSE\u0026amp;#34; | jq -r --arg name \u0026amp;#34;$name\u0026amp;#34; \u0026amp;#39;[.dnsEntries[] | select(.name==$name and .type==\u0026amp;#34;TXT\u0026amp;#34;)]\u0026amp;#39;)\n\n    # Check if there are existing records that need to be removed\n    MATCHING_COUNT=$(echo \u0026amp;#34;$MATCHING_RECORDS\u0026amp;#34; | jq \u0026amp;#39;length\u0026amp;#39;)\n\n    if [ \u0026amp;#34;$MATCHING_COUNT\u0026amp;#34; -eq 0 ]; then\n        log_message \u0026amp;#34;No existing TXT record found for $name to remove.\u0026amp;#34;\n    else\n        log_message \u0026amp;#34;Removing all existing TXT records for $name.\u0026amp;#34;\n\n        # Loop through all found records and remove them\n        echo \u0026amp;#34;$MATCHING_RECORDS\u0026amp;#34; | jq -c \u0026amp;#39;.[]\u0026amp;#39; | while read -r record; do\n            existing_name=$(echo \u0026amp;#34;$record\u0026amp;#34; | jq -r \u0026amp;#39;.name\u0026amp;#39;)\n            existing_expire=$(echo \u0026amp;#34;$record\u0026amp;#34; | jq -r \u0026amp;#39;.expire\u0026amp;#39;)\n            existing_content=$(echo \u0026amp;#34;$record\u0026amp;#34; | jq -r \u0026amp;#39;.content\u0026amp;#39;)\n\n            # Make the DELETE call using the full details\n            DELETE_ENTRY=$(jq -n \\\n            --arg name \u0026amp;#34;$existing_name\u0026amp;#34; \\\n            --arg expire \u0026amp;#34;$existing_expire\u0026amp;#34; \\\n            --arg type \u0026amp;#34;TXT\u0026amp;#34; \\\n            --arg content \u0026amp;#34;$existing_content\u0026amp;#34; \\\n            \u0026amp;#39;{\u0026amp;#34;dnsEntry\u0026amp;#34;: {\u0026amp;#34;name\u0026amp;#34;: $name, \u0026amp;#34;expire\u0026amp;#34;: ($expire | tonumber), \u0026amp;#34;type\u0026amp;#34;: $type, \u0026amp;#34;content\u0026amp;#34;: $content}}\u0026amp;#39;)\n\n            DELETE_RESPONSE=$(curl -s -X DELETE \\\n                -H \u0026amp;#34;Authorization: Bearer $ACCESS_TOKEN\u0026amp;#34; \\\n                -H \u0026amp;#34;Content-Type: application/json\u0026amp;#34; \\\n                -d \u0026amp;#34;$DELETE_ENTRY\u0026amp;#34; \\\n                \u0026amp;#34;https://api.transip.nl/v6/domains/$DOMAIN/dns\u0026amp;#34;)\n\n            # Check if the removal was successful\n            if [[ \u0026amp;#34;$DELETE_RESPONSE\u0026amp;#34; == *\u0026amp;#34;error\u0026amp;#34;* ]]; then\n                log_message \u0026amp;#34;ERROR: The TXT record $existing_name could not be removed.\u0026amp;#34;\n            else\n                log_message \u0026amp;#34;TXT record $existing_name successfully removed.\u0026amp;#34;\n            fi\n        done\n    fi\n}\n\n# Always generate a new token for each run\ngenerate_new_token\n\n# Certbot passes these values when adding and removing the record\nif [ \u0026amp;#34;$1\u0026amp;#34; == \u0026amp;#34;add\u0026amp;#34; ]; then\n    add_or_update_txt_record \u0026amp;#34;$CERTBOT_DOMAIN\u0026amp;#34; \u0026amp;#34;$CERTBOT_VALIDATION\u0026amp;#34;\n    sleep 25  # Wait for DNS update to propagate\nelif [ \u0026amp;#34;$1\u0026amp;#34; == \u0026amp;#34;clean\u0026amp;#34; ]; then\n    remove_txt_record \u0026amp;#34;$CERTBOT_DOMAIN\u0026amp;#34;\nfi\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch2 id=\"steps-to-run\"\u003eSteps to run\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eDownload the Private Key\u003c/strong\u003e: Save the private key file (\u003ccode\u003etransip_private_key.pem\u003c/code\u003e) from TransIP’s API settings somewhere on your device you are going to run this from.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSet Up Docker Compose\u003c/strong\u003e: Place the \u003ccode\u003edocker-compose.yml\u003c/code\u003e and \u003ccode\u003ecertbot-transip.sh\u003c/code\u003e files in the same directory. (or edit the script)\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eRun the Container\u003c/strong\u003e: Start the container (cd to the path where your yml is)\u003c/li\u003e\n\u003c/ol\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003edocker-compose up --build\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eMonitor the Log output: The log output of the docker container will show the progress of DNS challenges and certificate issuance.\u003c/p\u003e\n\u003cp\u003eThats all there is to it!\u003c/p\u003e\n\u003cp\u003eI will also post a version with a working helm chart soon. This was a little bigger culprit to make sure it all works correctly and updates all certificates through different namespaces where i needed the certificates to be available on different domains.\u003c/p\u003e\n","date_modified":"2024-11-29T16:52:31+01:00","date_published":"2024-11-28T22:03:00+01:00","id":"https://www.allsubjectsmatter.nl/automating-lets-encrypt-with-certbot-and-transip-using-docker-2/","language":"en","summary":"Want to auto update/manage your SSL certificates inside your Docker with Certbot and hosting your domain with TransIP? Then look no further!","tags":["Under the hood"],"title":"Automating Let's Encrypt with Certbot and TransIP using Docker","url":"https://www.allsubjectsmatter.nl/automating-lets-encrypt-with-certbot-and-transip-using-docker-2/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eThe API: \u003ca href=\"https://api.protonvpn.ch/vpn/logicals\" data-external\u003ehttps://api.protonvpn.ch/vpn/logicals\u003c/a\u003e has been formally shut down according to Proton Support. The only thing it returns now is:\u003c/p\u003e\n\u003cpre class=\"line-numbers language-json\"\u003e\u003ccode class=\"language-json\"\u003e{\u0026amp;#34;Code\u0026amp;#34;:0,\u0026amp;#34;Error\u0026amp;#34;:\u0026amp;#34;Invalid access token\u0026amp;#34;,\u0026amp;#34;Details\u0026amp;#34;:{}}\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eSo this post is sadly obsolete now. Somebody started a petition to get this feature back.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://protonmail.uservoice.com/forums/932836-proton-vpn/suggestions/49774472-re-enable-access-to-api\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eRe-enable access to API\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eUntil recently, users could access the API (https://api.protonvpn.ch/vpn/logicals) to filter servers based on criteria such as country and server load, to make sure they connect to a server that isn\u0026#39;t overloaded. This ability was removed. 27 votes, 4 comments.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cspan\u003eProton VPN feedback\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"the-deprecated-guide-which-no-longer-functions-as-of-062025-for-historical-reasons\"\u003eThe deprecated guide which no longer functions as of 06/2025 For historical reasons.\u003c/h2\u003e\n\u003cp\u003eWhen ProtonVPN discontinued its command-line interface (CLI) tool, users of headless systems like Raspberry Pi were left without an easy method to manage VPN connections. Without a graphical user interface, manually switching VPN servers has become a cumbersome task. Also selecting the fastest VPN in a pool of servers has become impossible.\u003c/p\u003e\n\u003cp\u003eTo solve this problem, I developed a Python script that automatically selects the fastest ProtonVPN server based on latency and load, then updates your WireGuard configuration for you. This guide will walk you through setting up WireGuard for ProtonVPN, installing the script, and automating your VPN connection to ensure your Raspberry Pi is always connected to the best server.\u003c/p\u003e\n\u003ch3 id=\"why-do-i-need-this-protonss-script\"\u003eWhy do i need this Protonss script?\u003c/h3\u003e\n\u003cp\u003eThis Python script automates the process of connecting your Raspberry Pi to the fastest ProtonVPN server available, using \u003cstrong\u003eWireGuard\u003c/strong\u003e as the VPN protocol. It checks the latency and load of multiple VPN servers from ProtonVPN\u0026rsquo;s API, selects the one with the best performance, updates your WireGuard configuration with the new server information, and reconnects your VPN only if a significantly faster server is found (nobody actually likes reconnects). This script also logs the entire process for easier troubleshooting.\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eNo more manual VPN switching\u003c/strong\u003e: Without ProtonVPN\u0026rsquo;s CLI, switching to the fastest server would require manual work. This script automates the process by regularly checking the available servers and updating the connection based on real-time performance.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eOptimizes connection speed\u003c/strong\u003e: The script ensures you’re always connected to the VPN server with the lowest latency and load. This is especially useful for maintaining high-speed connections in countries with fast or low-traffic ProtonVPN servers.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eReduces downtime\u003c/strong\u003e: By automating the reconnection process, the script helps minimize potential downtimes caused by poor server performance or slow connections.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eTailored for Raspberry Pi\u003c/strong\u003e: If you\u0026rsquo;re running a headless Raspberry Pi (without a monitor), this script is perfect for automating VPN tasks without requiring constant oversight.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSecurity \u0026amp; Privacy\u003c/strong\u003e: By ensuring your Raspberry Pi always connects to a VPN, it helps protect your data and online activity, especially if you\u0026rsquo;re using the Pi as a torrenting or networking device.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eIn short, this script automates finding the optimal ProtonVPN server, providing faster speeds, more reliability, and convenience for headless systems like a Raspberry Pi.\u003c/p\u003e\n\u003ch2 id=\"lets-dive-in\"\u003eLet\u0026rsquo;s dive in!\u003c/h2\u003e\n\u003cp\u003eFirst, you need to install WireGuard, which is a fast, lightweight VPN protocol which we will use to connect to our ProtonVPN server.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003e# First, update your package list and install WireGuard:\nsudo apt update\nsudo apt install wireguard curl resolvconf\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003e\u003cstrong\u003eGenerate a WireGuard configuration from ProtonVPN\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eLog in to your ProtonVPN account on their website.\u003c/li\u003e\n\u003cli\u003eGo to the \u0026ldquo;Downloads\u0026rdquo; section in your dashboard.\u003c/li\u003e\n\u003cli\u003eGenerate a new configuration file under \u0026ldquo;WireGuard Configurations.\u0026rdquo;\u003c/li\u003e\n\u003cli\u003eIf you\u0026rsquo;re using torrents, enable \u003cstrong\u003eNAT-PMP\u003c/strong\u003e.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/supercharge-your-headless-raspberry-pi-vpn-with-protonvpn-and-wireguard/images/image_hu_e1bb7171adce48a.webp\" alt=\"\" width=\"560\" height=\"368\" data-zoom-src=\"https://www.allsubjectsmatter.nl/supercharge-your-headless-raspberry-pi-vpn-with-protonvpn-and-wireguard/images/image_hu_1345a69245a3f8c.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003cp\u003eSave this configuration file on your Raspberry Pi:\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003e# Save the configuration file to your Raspberry Pi:\n# Place it in the WireGuard directory and rename it for simplicity:\nsudo nano /etc/wireguard/protonvpn.conf\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eOnce you\u0026rsquo;ve placed the configuration file in the correct directory, verify the connection.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003e# Verify connection to ProtonVPN:\nsudo wg-quick up protonvpn\n\n# This should give an ip address different to the one frome your isp\ncurl ifconfig.me \n\n# Verify connection to ProtonVPN:\nsudo wg-quick down protonvpn\n\ncurl ifconfig.me \u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eIf these two ip addresses from ifconfig.me are different your are connected to a VPN. even if you dont know your ip address since its a DHCP address or other.\u003c/p\u003e\n\u003ch3 id=\"installing-our-python-environment-for-protonss\"\u003eInstalling our Python environment for Protonss.\u003c/h3\u003e\n\u003cp\u003eWhen developing and running Python scripts on your Raspberry Pi, it\u0026rsquo;s best to use a \u003cstrong\u003ePython virtual environment\u003c/strong\u003e. A virtual environment allows you to install and manage Python packages in an isolated setting, ensuring that your system’s default Python configuration remains untouched.\u003c/p\u003e\n\u003cp\u003eThis way, any changes made to your environment will only affect the project you\u0026rsquo;re working on—such as installing specific packages like \u003ccode\u003erequests\u003c/code\u003e—without causing conflicts with other Python projects or system-level dependencies.\u003c/p\u003e\n\u003cp\u003eHere’s how you can set up and use a virtual environment for our script:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eInstall virtual environment\u003c/strong\u003e: First, install the \u003ccode\u003epython3-venv\u003c/code\u003e package, which allows you to create and manage virtual environments.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo apt install -y python3-venv python3-full\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003col start=\"2\"\u003e\n\u003cli\u003e\u003cstrong\u003eCreate a virtual environment\u003c/strong\u003e: Once installed, you can create a virtual environment. This will generate an isolated environment in your specified directory (in this case, \u003ccode\u003e~/protonss\u003c/code\u003e).\u003c/li\u003e\n\u003c/ol\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003epython3 -m venv ~/protonss\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003col start=\"3\"\u003e\n\u003cli\u003e\u003cstrong\u003eActivate the virtual environment\u003c/strong\u003e: After creating the environment, you’ll need to activate it. This tells your system to use the Python and packages from the virtual environment, instead of the system-wide Python installation.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esource ~/protonss/bin/activate\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cul\u003e\n\u003cli\u003eNow that the environment is active, any Python package you install (such as \u003ccode\u003erequests\u003c/code\u003e) will be installed into this isolated environment.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003epip install requests\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003e\u003cstrong\u003eUsing this virtual environment in the future\u003c/strong\u003e: Each time you want to run Python scripts that depend on this virtual environment, you’ll need to activate it again using the \u003ccode\u003esource ~/venv/bin/activate\u003c/code\u003e command. Once activated, any Python commands or packages you use will run in this environment.\u003c/p\u003e\n\u003ch3 id=\"installing-our-protonss-script-to-automate-a-connection-to-the-fastest-vpn\"\u003eInstalling our ProtonSS script to automate a connection to the fastest VPN.\u003c/h3\u003e\n\u003cp\u003eWe will create a configuration file to hold all the variables that the Python script will use. For example, you can define the path to your WireGuard configuration, log file, and the number of pings to test latency.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003ecd ~/protonss\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eSample \u003ccode\u003econfig.ini\u003c/code\u003e:\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003enano config.ini\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cpre class=\"line-numbers language-ini\"\u003e\u003ccode class=\"language-ini\"\u003e[Settings]\n# Path to your WireGuard configuration file\nwg_config_path = /etc/wireguard/protonvpn.conf\n\n# Path to your log file\nlog_file = /var/log/wireguard_updater.log\n\n# Number of pings to measure latency\nping_count = 3\n\n# Maximum score difference before reconnection 0 - 100\n# Average score of servers is between 10-40 it seems. so setting this to high results in no switches from servers\nscore_threshold = 5.0\n\n# Comma-separated list of preferred countries\npreferred_countries = US\n\n# Comma-separated list of preferred cities\n# If specified, servers must be in these cities (unless they\u0026amp;#39;re in a country without any specified city)\n# Preferred cities to connect to. If specified, servers must be in these cities\n# (unless they\u0026amp;#39;re in a country without any specified city).\n# Example: preferred_cities = Phoenix, Los Angeles, Seattle\n# This allows you to target specific cities within your preferred countries.\n# If left empty, any city within the preferred countries will be considered.\n# Note: City names are case sensitive and must match ProtonVPN\u0026amp;#39;s naming exactly.\n# Multiple cities should be comma-separated.\npreferred_cities =Phoenix\n\n# Weight for latency in the server score\n# latency is subjective since this is probably measured through the vpn and only counts as server latency.\n# Checking the latency without a vpn isnt recommended since the proton servers wont reply back without vpn.\n# Value cant be higher then 1. 0.3 translates to 30% of the score_treshold.\nlatency_weight = 0.3\n\n# Weight for server load in the server score\n# Load is the best factor 0.7 translates to 70% of the weight of the score_threshold. this cant be higher then 1.\nload_weight = 0.7\n\n# Enable or disable pinging servers (True or False)\n# Since ping might not be reliable enough for you to keep on switching servers this can be disabled here.\nping_enabled = True\n\n# Specify the interface to use for ping tests. By setting this option you can ping to the fastest vpn server\n# this will ensure the lowest latency VPN from your location.\nping_interface = eth0\n\n[Features]\n# Choose which type of servers you want to use\n# Options: p2p, plus, secure_core, tor\n# Default is p2p if not specified\nfeature_type = p2p\n\n# Feature explanation:\n# Each server has exactly ONE feature number:\n# 1, 17 = Secure Core servers\n# 2 = TOR servers\n# 0, 8, 16, 24 = Plus-only servers\n# 4, 12, 20, 28 = P2P servers (these are also Plus servers)\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eYou can customize the variables as needed. For example, leaving \u003ccode\u003epreferred_countries\u003c/code\u003e empty will make the script check a LOT! of servers, which will make the script run a very very long time!\u003c/p\u003e\n\u003ch4 id=\"create-the-python-script\"\u003e\u003cstrong\u003eCreate the Python script\u003c/strong\u003e\u003c/h4\u003e\n\u003cp\u003eNow, we are going to create the Python script that will automate your VPN connection. We first need to prepare a few steps\u003c/p\u003e\n\u003cp\u003eThe first line of the script should be a \u003cstrong\u003eshebang\u003c/strong\u003e, which ensures the script always runs in the correct Python environment:\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esource ~/protonss/bin/activate\nwhich python\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eThis will provide you with the pad you need to put on the first rule of this script\u003c/p\u003e\n\u003cpre class=\"line-numbers language-text\"\u003e\u003ccode class=\"language-text\"\u003e/home/pi/venv/bin/python\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eModify the script by adding a \u003cstrong\u003eshebang\u003c/strong\u003e: Add the following line at the very top of your Python script. Replace \u003ccode\u003e/home/{user}/venv/bin/python\u003c/code\u003e with the actual path you got when you ran the \u003ccode\u003ewhich python\u003c/code\u003e command earlier. Make sure to keep the \u003ccode\u003e#!\u003c/code\u003e at the beginning of the line—this tells the system which interpreter to use to run the script. we need the absolute path so ~/ wont work.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003enano protonvpn_auto_connect.py\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cpre class=\"line-numbers language-python\"\u003e\u003ccode class=\"language-python\"\u003e#!/home/{user}/protonss/bin/python\n\nimport requests\nimport subprocess\nimport os\nimport sys\nimport configparser\nimport logging\nimport re\nfrom datetime import datetime\nimport math\nimport shutil\n\n# Get the directory of the current script\nscript_dir = os.path.dirname(os.path.abspath(__file__))\n\n# Load the config file from the same directory as the script\nconfig = configparser.ConfigParser()\nconfig.read(os.path.join(script_dir, \u0026amp;#39;config.ini\u0026amp;#39;))\n\nWG_CONFIG_PATH = config.get(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;wg_config_path\u0026amp;#39;)\nLOG_FILE = config.get(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;log_file\u0026amp;#39;)\nPING_COUNT = config.getint(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;ping_count\u0026amp;#39;)\nSCORE_THRESHOLD = config.getfloat(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;score_threshold\u0026amp;#39;, fallback=5.0)\nPREFERRED_COUNTRIES = config.get(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;preferred_countries\u0026amp;#39;, fallback=\u0026amp;#39;\u0026amp;#39;).split(\u0026amp;#39;,\u0026amp;#39;)\nPREFERRED_CITIES = config.get(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;preferred_cities\u0026amp;#39;, fallback=\u0026amp;#39;\u0026amp;#39;).split(\u0026amp;#39;,\u0026amp;#39;)\nPING_INTERFACE = config.get(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;ping_interface\u0026amp;#39;, fallback=None)\n\n# Weighting factors for latency and load from the config\nLATENCY_WEIGHT = config.getfloat(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;latency_weight\u0026amp;#39;, fallback=0.3)\nLOAD_WEIGHT = config.getfloat(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;load_weight\u0026amp;#39;, fallback=0.7)\n\n# Ping option from config\nPING_ENABLED = config.getboolean(\u0026amp;#39;Settings\u0026amp;#39;, \u0026amp;#39;ping_enabled\u0026amp;#39;, fallback=True)\n\n# Clean up country codes (remove spaces and convert to uppercase)\nPREFERRED_COUNTRIES = [country.strip().upper() for country in PREFERRED_COUNTRIES if country.strip()]\n\n# Clean up city names (remove spaces)\nPREFERRED_CITIES = [city.strip() for city in PREFERRED_CITIES if city.strip()]\n\n# Set up logging\nlogging.basicConfig(\n    filename=LOG_FILE,\n    level=logging.INFO,\n    format=\u0026amp;#39;%(asctime)s [%(levelname)s] %(message)s\u0026amp;#39;,\n    datefmt=\u0026amp;#39;%Y-%m-%d %H:%M:%S\u0026amp;#39;\n)\n\nlogging.info(\u0026amp;#39;--- ProtonVPN Auto Connect Script Start ---\u0026amp;#39;)\n\n# Feature definitions\nFEATURE_SECURE_CORE = [1, 17]  # Secure Core servers\nFEATURE_TOR = [2]         # TOR servers\nFEATURE_PLUS = [0, 8, 16, 24]  # Plus-only servers\nFEATURE_P2P = [4, 12, 20, 28]    # P2P servers (these are also Plus servers)\n# check https://api.protonmail.ch/vpn/logicals for more info. \n# be aware that the secure core and tor server are very limited so check if one is available for your country as stated in the config.ini file.\n# FREE servers are not included in the api response. so they are already left out.\n\ndef get_feature_description(features):\n    # If no feature type is specified in config.ini, default to P2P\n    # Features is a number representing server capabilities\n    if not features:  # If features is 0 or None\n        return \u0026amp;#34;P2P\u0026amp;#34;  # Default to P2P\n    elif features in FEATURE_SECURE_CORE:\n        return \u0026amp;#34;Secure Core\u0026amp;#34;\n    elif features in FEATURE_TOR:\n        return \u0026amp;#34;TOR\u0026amp;#34; \n    elif features in FEATURE_P2P:\n        return \u0026amp;#34;P2P\u0026amp;#34;\n    elif features in FEATURE_PLUS:\n        return \u0026amp;#34;Plus\u0026amp;#34;\n    else:\n        return \u0026amp;#34;P2P\u0026amp;#34;  # Default to P2P for any unrecognized feature\n\ndef get_wanted_features():\n    \u0026amp;#34;\u0026amp;#34;\u0026amp;#34;Get the wanted feature type based on config priorities\u0026amp;#34;\u0026amp;#34;\u0026amp;#34;\n    feature_type = config.get(\u0026amp;#39;Features\u0026amp;#39;, \u0026amp;#39;feature_type\u0026amp;#39;, fallback=\u0026amp;#39;p2p\u0026amp;#39;).lower()\n    \n    if feature_type == \u0026amp;#39;secure_core\u0026amp;#39;:\n        return FEATURE_SECURE_CORE\n    elif feature_type == \u0026amp;#39;tor\u0026amp;#39;:\n        return FEATURE_TOR\n    elif feature_type == \u0026amp;#39;plus\u0026amp;#39;:\n        return FEATURE_PLUS\n    else:  # Default to P2P\n        return FEATURE_P2P\n\n# Function to fetch the list of servers from the ProtonVPN API\ndef get_servers(full_list=False):\n    try:\n        url = \u0026amp;#39;https://api.protonmail.ch/vpn/logicals\u0026amp;#39;\n        response = requests.get(url)\n        response.raise_for_status()\n        data = response.json()\n        logging.info(\u0026amp;#34;API response data received.\u0026amp;#34;)\n        return data[\u0026amp;#39;LogicalServers\u0026amp;#39;]\n    except requests.RequestException as e:\n        logging.error(f\u0026amp;#39;Error fetching server list: {e}\u0026amp;#39;)\n        sys.exit(1)\n    except ValueError as e:\n        logging.error(f\u0026amp;#39;Error parsing JSON: {e}\u0026amp;#39;)\n        sys.exit(1)\n\n# Function to filter P2P servers and servers in preferred countries\ndef filter_servers(servers):\n    filtered_servers = []\n    \n    # Get feature preferences from config (nu maar één type)\n    wanted_features = get_wanted_features()\n    \n    logging.info(f\u0026amp;#34;Filtering servers with preferences - Countries: {PREFERRED_COUNTRIES or \u0026amp;#39;Any\u0026amp;#39;}, \u0026amp;#34;\n                f\u0026amp;#34;Cities: {PREFERRED_CITIES or \u0026amp;#39;Any\u0026amp;#39;}, Features: {[get_feature_description(f) for f in wanted_features]}\u0026amp;#34;)\n    \n    for server in servers:\n        if not isinstance(server, dict):\n            logging.error(f\u0026amp;#34;Unexpected type for server: {type(server)} - Value: {server}\u0026amp;#34;)\n            continue\n            \n        features = server.get(\u0026amp;#39;Features\u0026amp;#39;, 0)\n        country = server[\u0026amp;#39;ExitCountry\u0026amp;#39;]\n        city = server.get(\u0026amp;#39;City\u0026amp;#39;, \u0026amp;#39;\u0026amp;#39;)\n\n        # Check if server features match our wanted features\n        if features in wanted_features:\n            # Check country and city preferences\n            if country in PREFERRED_COUNTRIES:\n                if not PREFERRED_CITIES or city in PREFERRED_CITIES:\n                    filtered_servers.append(server)\n                    logging.info(f\u0026amp;#34;Server matched - Country: {country}, City: {city}, \u0026amp;#34;\n                              f\u0026amp;#34;Features: {get_feature_description(features)}\u0026amp;#34;)\n            \n    if not filtered_servers:\n        logging.warning(f\u0026amp;#34;No servers found matching filters. Countries: {PREFERRED_COUNTRIES}, \u0026amp;#34;\n                      f\u0026amp;#34;Cities: {PREFERRED_CITIES}, Features: {[get_feature_description(f) for f in wanted_features]}\u0026amp;#34;)\n    else:\n        logging.info(f\u0026amp;#34;Found {len(filtered_servers)} matching servers\u0026amp;#34;)\n        \n    return filtered_servers\n\n# Function to measure latency by pinging the server (optional)\ndef get_latency(hostname):\n    if not PING_ENABLED:\n        logging.info(f\u0026amp;#34;Ping disabled, skipping latency check for {hostname}.\u0026amp;#34;)\n        return 0  # Return 0 latency if ping is disabled\n\n    latencies = []\n    for i in range(PING_COUNT):\n        try:\n            # Add the interface option if it\u0026amp;#39;s specified\n            ping_command = [\u0026amp;#39;ping\u0026amp;#39;, \u0026amp;#39;-c\u0026amp;#39;, \u0026amp;#39;1\u0026amp;#39;, \u0026amp;#39;-W\u0026amp;#39;, \u0026amp;#39;1\u0026amp;#39;, hostname]\n            if PING_INTERFACE:\n                ping_command.insert(1, \u0026amp;#39;-I\u0026amp;#39;)\n                ping_command.insert(2, PING_INTERFACE)\n\n            result = subprocess.run(ping_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)\n            if result.returncode == 0:\n                output = result.stdout.decode()\n                latency_line = [line for line in output.split(\u0026amp;#39;\\n\u0026amp;#39;) if \u0026amp;#39;time=\u0026amp;#39; in line][0]\n                latency = float(latency_line.split(\u0026amp;#39;time=\u0026amp;#39;)[1].split(\u0026amp;#39; ms\u0026amp;#39;)[0])\n                latencies.append(latency)\n            else:\n                latencies.append(float(\u0026amp;#39;inf\u0026amp;#39;))\n        except Exception as e:\n            latencies.append(float(\u0026amp;#39;inf\u0026amp;#39;))\n            logging.error(f\u0026amp;#39;Ping error for {hostname}: {e}\u0026amp;#39;)\n\n    # Return the lowest (best) latency value\n    best_latency = min(latencies)\n    \n    # Handle inf latency\n    if math.isinf(best_latency):\n        logging.warning(f\u0026amp;#34;Latency for {hostname} is inf. Assigning high latency value.\u0026amp;#34;)\n        best_latency = 1000  # Assign a high latency value\n\n    return best_latency\n\n# Function to select the best server based on a weighted combination of latency and load\ndef select_best_server(servers):\n    best_server = None\n    best_score = float(\u0026amp;#39;inf\u0026amp;#39;)\n    \n    for server in servers:\n        server_load = server.get(\u0026amp;#39;Load\u0026amp;#39;)\n        \n        # Handle inf load\n        if server_load is None or math.isinf(server_load):\n            logging.warning(f\u0026amp;#34;Server {server[\u0026amp;#39;Name\u0026amp;#39;]} has inf load. Assigning high load value.\u0026amp;#34;)\n            server_load = 100  # Assign a high load value if it\u0026amp;#39;s inf or None\n        else:\n            server_load = int(server_load)\n        \n        for srv in server[\u0026amp;#39;Servers\u0026amp;#39;]:\n            exit_ip = srv.get(\u0026amp;#39;ExitIP\u0026amp;#39;)\n            hostname = exit_ip\n\n            # Get latency, only if ping is enabled\n            latency = get_latency(hostname)\n\n            # Combine latency and load using the configured weights\n            score = (latency * LATENCY_WEIGHT) \u0026#43; (server_load * LOAD_WEIGHT)\n            \n            logging.info(f\u0026amp;#34;Server {server[\u0026amp;#39;Name\u0026amp;#39;]} - Latency: {latency:.2f} ms, Load: {server_load}%, Score: {score:.2f}\u0026amp;#34;)\n            \n            if score \u0026amp;lt; best_score:\n                best_score = score\n                best_server = server\n                best_server_instance = srv\n\n    return best_server, best_server_instance, best_score\n\n# Function to read the current server IP from the WireGuard configuration\ndef get_current_server_ip():\n    try:\n        with open(WG_CONFIG_PATH, \u0026amp;#39;r\u0026amp;#39;) as f:\n            config_text = f.read()\n        # Extract the current Endpoint IP\n        match = re.search(r\u0026amp;#39;Endpoint\\s*=\\s*([\\d\\.]\u0026#43;):\\d\u0026#43;\u0026amp;#39;, config_text)\n        if match:\n            logging.info(f\u0026amp;#34;Current server IP found in WireGuard configuration: {match.group(1)}\u0026amp;#34;)\n            return match.group(1)\n        else:\n            logging.warning(\u0026amp;#39;No current Endpoint IP found in WireGuard configuration.\u0026amp;#39;)\n            return None\n    except Exception as e:\n        logging.error(f\u0026amp;#39;Error reading the WireGuard configuration: {e}\u0026amp;#39;)\n        return None\n\n# Function to find the current server\u0026amp;#39;s performance in the live server data\ndef get_current_server_performance(servers, current_ip):\n    if current_ip is None:\n        logging.warning(\u0026amp;#39;Current server IP is None, skipping performance check.\u0026amp;#39;)\n        return None, None, float(\u0026amp;#39;inf\u0026amp;#39;)\n    \n    for server in servers:\n        for srv in server[\u0026amp;#39;Servers\u0026amp;#39;]:\n            if srv[\u0026amp;#39;ExitIP\u0026amp;#39;] == current_ip:\n                server_load = server.get(\u0026amp;#39;Load\u0026amp;#39;)\n                if server_load is None or math.isinf(server_load):\n                    logging.warning(f\u0026amp;#34;Server {server[\u0026amp;#39;Name\u0026amp;#39;]} (IP: {current_ip}) has inf load. Assigning high load value.\u0026amp;#34;)\n                    server_load = 100  # Assign a high load value\n                else:\n                    server_load = int(server_load)\n\n                latency = get_latency(current_ip) if PING_ENABLED else 0\n                score = (latency * LATENCY_WEIGHT) \u0026#43; (server_load * LOAD_WEIGHT)\n\n                logging.info(f\u0026amp;#34;Current server {server[\u0026amp;#39;Name\u0026amp;#39;]} - Latency: {latency:.2f} ms, Load: {server_load}%, Score: {score:.2f}\u0026amp;#34;)\n                return server, srv, score\n    logging.warning(f\u0026amp;#34;Current server with IP {current_ip} not found in live data.\u0026amp;#34;)\n    return None, None, float(\u0026amp;#39;inf\u0026amp;#39;)\n\n# Function to find the current server in the full ProtonVPN list if not found in filtered data\ndef find_current_server_full_list(current_ip):\n    servers = get_servers(full_list=True)\n    return get_current_server_performance(servers, current_ip)\n\n# Function to update the WireGuard configuration file\ndef update_wg_config(server, server_instance, best_score):\n    try:\n        endpoint_ip = server_instance[\u0026amp;#39;EntryIP\u0026amp;#39;]\n        endpoint_port = 51820  # Default WireGuard port\n        endpoint = f\u0026amp;#34;{endpoint_ip}:{endpoint_port}\u0026amp;#34;\n        public_key = server_instance[\u0026amp;#39;X25519PublicKey\u0026amp;#39;]\n        server_name = server[\u0026amp;#39;Name\u0026amp;#39;]\n\n        # Read the current WireGuard configuration\n        with open(WG_CONFIG_PATH, \u0026amp;#39;r\u0026amp;#39;) as f:\n            config_text = f.read()\n\n        # Strip comments that start with # under [Peer] before PublicKey\n        config_text = re.sub(r\u0026amp;#39;#.*\\n\u0026amp;#39;, \u0026amp;#39;\u0026amp;#39;, config_text, flags=re.M)\n\n        # Update the Endpoint in the configuration\n        config_text = re.sub(r\u0026amp;#39;Endpoint\\s*=.*\u0026amp;#39;, f\u0026amp;#39;Endpoint = {endpoint}\u0026amp;#39;, config_text)\n\n        # Update the PublicKey in the configuration\n        config_text = re.sub(r\u0026amp;#39;PublicKey\\s*=.*\u0026amp;#39;, f\u0026amp;#39;PublicKey = {public_key}\u0026amp;#39;, config_text)\n\n        # Add the new comment indicating the chosen server and score\n        config_text = re.sub(r\u0026amp;#39;\\[Peer\\]\\n\u0026amp;#39;, f\u0026amp;#39;[Peer]\\n# {server_name} - Auto-generated by ProtonVPN Auto Connect - Score: {best_score:.2f}\\n\u0026amp;#39;, config_text)\n\n        # Write the updated configuration back to the file\n        with open(WG_CONFIG_PATH, \u0026amp;#39;w\u0026amp;#39;) as f:\n            f.write(config_text)\n        os.chmod(WG_CONFIG_PATH, 0o600)\n\n        logging.info(f\u0026amp;#34;WireGuard configuration updated with server: {server_name} ({server[\u0026amp;#39;ExitCountry\u0026amp;#39;]}), Endpoint: {endpoint}, PublicKey: {public_key}, Score: {best_score:.2f}\u0026amp;#34;)\n    except Exception as e:\n        logging.error(f\u0026amp;#39;Error updating the WireGuard configuration: {e}\u0026amp;#39;)\n        sys.exit(1)\n\n# Function to restart the WireGuard interface\ndef is_resolvconf_available():\n    \u0026amp;#34;\u0026amp;#34;\u0026amp;#34;Check if resolvconf is available on the system.\u0026amp;#34;\u0026amp;#34;\u0026amp;#34;\n    return shutil.which(\u0026amp;#34;resolvconf\u0026amp;#34;) is not None\n\ndef is_wg_quick_available():\n    \u0026amp;#34;\u0026amp;#34;\u0026amp;#34;Check if wg-quick is available on the system.\u0026amp;#34;\u0026amp;#34;\u0026amp;#34;\n    return shutil.which(\u0026amp;#34;wg-quick\u0026amp;#34;) is not None\n\ndef restart_wg():\n    interface_name = os.path.basename(WG_CONFIG_PATH).split(\u0026amp;#39;.\u0026amp;#39;)[0]\n    \n    # Check if wg-quick is available before proceeding\n    if not is_wg_quick_available():\n        logging.error(\u0026amp;#34;wg-quick not found. Ensure WireGuard is installed correctly.\u0026amp;#34;)\n        sys.exit(1)\n\n    try:\n        # Check if resolvconf is available before proceeding\n        if not is_resolvconf_available():\n            logging.warning(\u0026amp;#34;resolvconf not found, DNS settings may not be updated correctly.\u0026amp;#34;)\n\n        # Bring down the interface\n        subprocess.run([\u0026amp;#39;wg-quick\u0026amp;#39;, \u0026amp;#39;down\u0026amp;#39;, interface_name], stdout=subprocess.PIPE, stderr=subprocess.PIPE)\n\n        # Bring up the interface\n        subprocess.run([\u0026amp;#39;wg-quick\u0026amp;#39;, \u0026amp;#39;up\u0026amp;#39;, interface_name], check=True)\n        logging.info(f\u0026amp;#39;WireGuard interface {interface_name} started.\u0026amp;#39;)\n\n    except subprocess.CalledProcessError as e:\n        logging.error(f\u0026amp;#39;Error starting the WireGuard interface: {e}\u0026amp;#39;)\n        sys.exit(1)\n\n# Main function\ndef main():\n    logging.info(\u0026amp;#39;--- Starting server selection process ---\u0026amp;#39;)\n    \n    # Fetch the list of servers\n    servers = get_servers()\n    if not isinstance(servers, list):\n        logging.error(f\u0026amp;#39;Unexpected type for servers: {type(servers)}\u0026amp;#39;)\n        sys.exit(1)\n\n    # Filter the servers\n    filtered_servers = filter_servers(servers)\n    if not filtered_servers:\n        logging.error(\u0026amp;#39;No suitable servers available after filtering.\u0026amp;#39;)\n        sys.exit(1)\n\n    # Get the current server\u0026amp;#39;s IP address from the WireGuard configuration\n    current_ip = get_current_server_ip()\n\n    # Get the current server\u0026amp;#39;s performance if it exists\n    current_server, current_server_instance, current_score = get_current_server_performance(filtered_servers, current_ip)\n\n    # If the current server is not found, check the full list of servers\n    if current_server is None:\n        logging.info(f\u0026amp;#34;Current server not found in filtered list, checking full server list...\u0026amp;#34;)\n        current_server, current_server_instance, current_score = find_current_server_full_list(current_ip)\n\n    # Select the best server based on weighted latency and load\n    best_server, best_server_instance, best_score = select_best_server(filtered_servers)\n    if not best_server:\n        logging.error(\u0026amp;#39;No suitable server found.\u0026amp;#39;)\n        sys.exit(1)\n\n    if current_server:\n        logging.info(f\u0026amp;#34;Current server {current_server[\u0026amp;#39;Name\u0026amp;#39;]} (IP: {current_ip}) - Score: {current_score:.2f}\u0026amp;#34;)\n    \n    logging.info(f\u0026amp;#34;Selected best server {best_server[\u0026amp;#39;Name\u0026amp;#39;]} - Score: {best_score:.2f}\u0026amp;#34;)\n    \n    if current_server and current_score \u0026amp;lt;= best_score \u0026#43; SCORE_THRESHOLD:\n        logging.info(f\u0026amp;#34;Current server {current_server[\u0026amp;#39;Name\u0026amp;#39;]} is within threshold. No switch necessary.\u0026amp;#34;)\n    else:\n        logging.info(f\u0026amp;#34;Switching to server {best_server[\u0026amp;#39;Name\u0026amp;#39;]} due to better performance.\u0026amp;#34;)\n        update_wg_config(best_server, best_server_instance, best_score)\n        restart_wg()\n\n    logging.info(\u0026amp;#39;--- ProtonVPN Auto Connect Script End ---\u0026amp;#39;)\n\nif __name__ == \u0026amp;#39;__main__\u0026amp;#39;:\n    main()\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch3 id=\"testing-the-script\"\u003eTesting the script\u003c/h3\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003echmod \u0026#43;x protonvpn_auto_connect.py\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eOnce the script is saved, give it execution permissions and run it\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo ./protonvpn_auto_connect.py\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003ecat /var/log/wireguard_updater.log\ntail -f /var/log/wireguard_updater.log\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eyou can tail or cat the log file to check its output in the logs. Tail is preferred since you can keep watching it while running your script. Just open a second ssh with the tail to keep reviewing it.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-log\"\u003e\u003ccode class=\"language-log\"\u003e2024-09-25 22:19:10 [INFO] --- ProtonVPN Auto Connect Script Start ---\n2024-09-25 22:19:10 [INFO] --- Starting server selection process ---\n2024-09-25 22:19:11 [INFO] API response data received.\n2024-09-25 22:19:11 [INFO] Current server IP found in WireGuard configuration: 79.135.105.20\n2024-09-25 22:19:11 [WARNING] Current server with IP 79.135.105.20 not found in filtered data.\n2024-09-25 22:19:11 [INFO] Current server not found in filtered list, checking full server list...\n2024-09-25 22:19:12 [INFO] API response data received.\n2024-09-25 22:19:13 [INFO] Current server BH#1 - Latency: 26.90 ms, Load: 10%, Score: 15.07\n2024-09-25 22:19:14 [INFO] Server AO#1 - Latency: 27.10 ms, Load: 7%, Score: 13.03\n2024-09-25 22:19:15 [INFO] Server AO#2 - Latency: 26.60 ms, Load: 14%, Score: 17.78\n2024-09-25 22:19:16 [INFO] Server AO#3 - Latency: 27.10 ms, Load: 9%, Score: 14.43\n2024-09-25 22:19:17 [INFO] Server AO#4 - Latency: 26.60 ms, Load: 16%, Score: 19.18\n2024-09-25 22:19:17 [INFO] Current server BH#1 (IP: 79.135.105.20) - Score: 15.07\n2024-09-25 22:19:17 [INFO] Selected best server AO#1 - Score: 13.03\n2024-09-25 22:19:17 [INFO] Current server BH#1 is within threshold. No switch necessary.\n2024-09-25 22:19:17 [INFO] --- ProtonVPN Auto Connect Script End ---\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo cat /etc/wireguard/protonvpn.conf\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eYou can also verify that everything works by checkin the protonvpn.conf file or by checking your external ip address.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-ini\"\u003e\u003ccode class=\"language-ini\"\u003e[Peer]\n# BH#3 - Auto-generated by ProtonVPN Auto Connect - Score: 7.74\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eyou will see something like this above. This comment will update when it connects to another VPN and the ipaddress inside the peer also changes with it off course.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003ecurl ifconfig.me #this wil give the external ip in the cli.\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eAs an additional step you can also check your external ip. this should be something different then the ip address provided by your provider.\u003c/p\u003e\n\u003ch3 id=\"running-this-script-every-hour\"\u003eRunning this script every hour.\u003c/h3\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo crontab -e\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cpre class=\"line-numbers language-text\"\u003e\u003ccode class=\"language-text\"\u003e0 * * * * cd /home/{user}/protonss \u0026amp;amp;\u0026amp;amp; /home/{user}/protonss/protonvpn_auto_connect.py \u0026amp;gt;\u0026amp;gt; /home/{user}/protonss/protonvpn_cron.log 2\u0026amp;gt;\u0026amp;amp;1\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eHere we defined it to run every hour. but can you schedule it how you want.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo crontab -l\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eSave the crontab and verify the cron is installed.\u003c/p\u003e\n\u003ch3 id=\"optional-block-specific-traffic-if-your-vpn-fails\"\u003eOptional: Block specific traffic if your VPN fails\u003c/h3\u003e\n\u003cp\u003eIf you want to ensure no traffic leaves your device if the VPN connection fails, you can use \u003ccode\u003eiptables\u003c/code\u003e to block traffic when the WireGuard interface is down.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo apt-get install iptables\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eVerify that you have installed iptables or install it with the above command.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003eifconfig -a\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cpre class=\"line-numbers language-text\"\u003e\u003ccode class=\"language-text\"\u003eprotonvpn: flags=209\u0026amp;lt;UP,POINTOPOINT,RUNNING,NOARP\u0026amp;gt;  mtu 1420\n        inet 10.2.1.3  netmask 255.255.255.255  destination 10.2.1.3\n        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)\n        RX packets 612  bytes 101768 (99.3 KiB)\n        RX errors 0  dropped 0  overruns 0  frame 0\n        TX packets 765  bytes 101144 (98.7 KiB)\n        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eFirst check the name of your network interface. If you named your file protonvpn.conf it should be named protonvpn.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo iptables -A OUTPUT -p tcp --dport 51413 ! -o protonvpn -j REJECT --reject-with icmp-port-unreachable\nsudo iptables -A OUTPUT -p udp --dport 51413 ! -o protonvpn -j REJECT --reject-with icmp-port-unreachable\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eyou can add any rule to your iptables. The 2 rules below make sure this port 51413 is rejected when the network interface called protonvpn isn\u0026rsquo;t available. There is a rule for UDP and TCP traffic in place.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo iptables -L OUTPUT -v -n\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eAfter adding all your required rules you can verify them with this command .\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo apt-get install iptables-persistent\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eAnd make these rules persistent so you dont have to add them on every reboot.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo netfilter-persistent save\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch3 id=\"conclusion\"\u003eConclusion\u003c/h3\u003e\n\u003cp\u003eWith this Python script, your Raspberry Pi will always be connected to the fastest ProtonVPN server available, ensuring you have the best possible VPN performance without manual intervention. Whether you\u0026rsquo;re using your Pi for networking, torrenting, or general online security, this automation ensures optimal performance and reliable privacy protection.\u003c/p\u003e\n\u003ch3 id=\"update20-12-2024\"\u003eUpdate - 20-12-2024\u003c/h3\u003e\n\u003cp\u003eAdded Preferred_cities as an option to shorten the US country select. That included over 2000 servers to check otherwise. The performance check can at least get a bit shorter this way :-).\u003c/p\u003e\n\u003cp\u003eAnd i found out what each Feature section for every server in the API includes. I will explain this below. I adjusted my code accordingly to work with this correctly. Be aware that the config you made earlier should reflect the type of server you are connecting to. If your Configuration isnt configured to connect to a Secure Core dont select a Secure Core in this script also. Reflect to the created ProtonVPN Configuration.\u003c/p\u003e\n\u003cp\u003eFeature explanation:\u003c/p\u003e\n\u003cp\u003eEach server has exactly ONE feature number:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e1, 17 = Secure Core servers\u003c/li\u003e\n\u003cli\u003e2 = TOR servers\u003c/li\u003e\n\u003cli\u003e0, 8, 16, 24 = Plus-only servers\u003c/li\u003e\n\u003cli\u003e4, 12, 20, 28 = P2P servers (these are also Plus servers it seems)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cpre class=\"line-numbers language-json\"\u003e\u003ccode class=\"language-json\"\u003e        {\n            \u0026amp;#34;Name\u0026amp;#34;: \u0026amp;#34;US-AZ#105\u0026amp;#34;,\n            \u0026amp;#34;EntryCountry\u0026amp;#34;: \u0026amp;#34;US\u0026amp;#34;,\n            \u0026amp;#34;ExitCountry\u0026amp;#34;: \u0026amp;#34;US\u0026amp;#34;,\n            \u0026amp;#34;Domain\u0026amp;#34;: \u0026amp;#34;node-us-260.protonvpn.net\u0026amp;#34;,\n            \u0026amp;#34;Tier\u0026amp;#34;: 2,\n            \u0026amp;#34;Features\u0026amp;#34;: 12,\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eAfter some more investigation regarding questions about the Free servers from ProtonVPN, it appears that ProtonVPN has removed public API access to their free server endpoints. The /vpn/servers endpoint that previously should have provided information about free servers now returns a 404 error:\u003c/p\u003e\n\u003cpre class=\"line-numbers language-json\"\u003e\u003ccode class=\"language-json\"\u003e{    \u0026amp;#34;Code\u0026amp;#34;: 404,    \u0026amp;#34;Error\u0026amp;#34;: \u0026amp;#34;Path not found\u0026amp;#34;,    \u0026amp;#34;Details\u0026amp;#34;: {}}\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eCurrently, only the Plus server endpoint is accessible via the public\u003c/p\u003e\n\u003cp\u003eAPI:url = \u0026lsquo;\u003ca href=\"https://api.protonmail.ch/vpn/logicals\" data-external\u003ehttps://api.protonmail.ch/vpn/logicals\u003c/a\u003e\u0026rsquo;\u003c/p\u003e\n\u003cp\u003eThis is likely a deliberate change by ProtonVPN. So sadly this script cant support the Free servers.\u003c/p\u003e\n","date_modified":"2025-09-01T09:21:58+02:00","date_published":"2024-09-25T22:24:00+02:00","id":"https://www.allsubjectsmatter.nl/supercharge-your-headless-raspberry-pi-vpn-with-protonvpn-and-wireguard/","language":"en","summary":"Deprecated: Effortlessly manage ProtonVPN on your headless device like a Raspberry Pi with this Python script. Automatically connect to the fastest server using WireGuard for optimal speed and security. This is to mitigate error 422 using the protonvpn-cli client which is broken as of now.","tags":["Under the hood"],"title":"Deprecated: Supercharge Your headless (Raspberry Pi) VPN with ProtonVPN and WireGuard","url":"https://www.allsubjectsmatter.nl/supercharge-your-headless-raspberry-pi-vpn-with-protonvpn-and-wireguard/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eA lot is being written about Scrum already. Scrum is dead and so on. Well when you have a vacancy for a \u0026ldquo;Manager\u0026rdquo; a \u0026ldquo;Janitor\u0026rdquo; or some other clearly non Scrum title but decide to call it a vacancy for a Product Owner or a Scrum Master. It\u0026rsquo;s you who are fucking up. just renaming the beast isn\u0026rsquo;t going to make it work like magic.\u003c/p\u003e\n\u003cp\u003eWe’ve all heard it: \u003cstrong\u003e\u0026ldquo;Let’s do Scrum, and Poof! We’ll deliver value in no time!\u0026rdquo;\u003c/strong\u003e (Spoiler alert) it doesn’t work like that. Scrum isn’t some magic potion that turns chaos into order just by sprinkling a few ceremonies around. It takes a lot more than following the steps in the Scrum guide to get to real results. It’s all about \u003cstrong\u003eteamwork, discipline, and having the right people in place\u003c/strong\u003eand believe it or not, it’s not that different from how things are trained in the army ;-).\u003c/p\u003e\n\u003ch2 id=\"sprint-equals-mission\"\u003eSprint equals Mission\u003c/h2\u003e\n\u003cp\u003eLet’s start once again with the elephant in the room, the word “sprint.” We’re not actually running a race, are we? This term still suggests rushing to me, but in reality, we’re aiming for a \u003cstrong\u003edeliberate, well-planned mission\u003c/strong\u003e. Like in the army, it’s not about speed it’s about precision and success. A well-executed mission isn’t just thrown together, it is strategically planned and relies on everyone playing their role and pulling their weight to the finish line.\u003c/p\u003e\n\u003cp\u003eDoing a mission requires adaptation along the way. You are going to encounter unknown\u0026rsquo;s and you need to be prepared to deal with them instead of avoid/ignoring them and fuckup your mission.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eSolving unknowns as a **\u003cu\u003eTEAM\u003c/u\u003e**is what\u0026rsquo;s going to make an Agile/Scrum approach work.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"specialized-but-all-round\"\u003eSpecialized, but all-round\u003c/h2\u003e\n\u003cp\u003eIn both Scrum teams and military units, we all have our own specialties and advantages. But here’s the kicker. We deliver value as a team. Just like the army, where soldiers are cross-trained to fill in gaps, \u003cstrong\u003ewe’re all-rounders too\u003c/strong\u003e. If something goes wrong, we don’t just throw our hands up and call it a day. We adapt and get it done, together (as a team!).\u003c/p\u003e\n\u003ch2 id=\"scrum-doesnt-work-without-the-right-mindset-scrum-doesne28099t-work-without-the-right-mindset\"\u003eScrum doesn’t work without the right mindset {#scrum-doesn%E2%80%99t-work-without-the-right-mindset}\u003c/h2\u003e\n\u003cp\u003eScrum doesn’t run itself. Just adopting Scrum won’t suddenly create value. I’ve seen it happen. People think if they run the ceremonies and just follow some Scrum related steps, magic will happen. But without \u003cstrong\u003eyour team managing the process itself\u003c/strong\u003e, it’s like trying to run an army mission with no captain, no briefing, and everyone just winging it. Sure, you might finish, but what have you actually done?\u003c/p\u003e\n\u003cp\u003eThe army knows this well. There’s a structure, leaders who guide, but also a reliance on every team member being proactive. In Scrum, you need the right mix: a \u003cstrong\u003eProduct Owner with a mandate and an actual vision that backs the company\u003c/strong\u003e, a \u003cstrong\u003eScrum Master who keeps the flow\u003c/strong\u003e going, and a \u003cstrong\u003eteam that’s not just doing tasks but thinking about the bigger picture\u003c/strong\u003e. You will achieve this mindset only by working together and keep talking as a single team.\u003c/p\u003e\n\u003ch2 id=\"a-team-that-talks-about-the-hard-stuff\"\u003eA team that talks about the hard stuff\u003c/h2\u003e\n\u003cp\u003eAs a team we need to work together, talk openly, and \u003cstrong\u003etackle the hard stuff\u003c/strong\u003e. I’m not talking about easy conversations where everyone agrees. I mean \u003cstrong\u003edifficult, uncomfortable discussions\u003c/strong\u003e about what’s not working or clear, what needs to improve, and how to get there. Only then right kind of delivered value is going to surface.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eDiscuss unknowns during your mission and get rid of them. Upholding a \u0026ldquo;daily scrum\u0026rdquo; wont magically teach that by itself.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eHow do you think Apple created the iPhone? Jobs and Ive didn’t sit around patting each other on the back. They argued, they iterated, they pushed each other until the product was \u003cstrong\u003eright\u003c/strong\u003e. And that’s how Scrum teams should operate. You don’t just do what’s easy or what’s fast. You work through the problems, challenge each other, and iterate until you’ve nailed it. You also ask a lot of (WHY) questions so you understand where each person is coming from.\u003c/p\u003e\n\u003ch2 id=\"briefings-and-debriefings-are-crucial-but-only-if-you-commit-to-them\"\u003eBriefings and debriefings are crucial, but only if you commit to them\u003c/h2\u003e\n\u003cp\u003eIn Scrum, we have our \u003cstrong\u003ePlanning (briefing)\u003c/strong\u003e and our \u003cstrong\u003eReview and Retrospective (debriefing)\u003c/strong\u003e. Sound familiar? You can’t just jump into the mission without a game plan, and you sure as hell can’t improve without looking back at what went wrong or right. But, and here’s the caveat \u003cstrong\u003eyou have to actually use these sessions\u003c/strong\u003e for what they’re intended. Too often, they’re rushed or treated as checkboxes. In the army, a failed briefing or debrief could cost lives, in Scrum, it’ll cost you the ability to iterate and test on things for the better the next mission. Take them seriously, or they’re just empty meetings not delivering any value.\u003c/p\u003e\n\u003ch2 id=\"teamwork-is-everything\"\u003eTeamwork is everything\u003c/h2\u003e\n\u003cp\u003eAt the end of the day, both Scrum and military missions succeed because of \u003cstrong\u003eteamwork\u003c/strong\u003e. You need people who are committed, who communicate, and who have each other’s backs. Without that, you’re just a bunch of people shouting the are doing Scrum, and that doesn’t deliver any value at all. Scrum’s not about heroes, it’s about the \u003cstrong\u003ecollective goal\u003c/strong\u003e. You win together, you lose together.\u003c/p\u003e\n\u003ch2 id=\"scrum-isnt-a-holy-hammer-that-solves-everything\"\u003eScrum isn\u0026rsquo;t a holy hammer that solves everything.\u003c/h2\u003e\n\u003cp\u003eSo, here’s the thing. \u003cstrong\u003eScrum is a framework\u003c/strong\u003e. It’s a humanly created structure designed to help teams work together efficiently. But just doing Scrum isn’t going to magically produce value for your company. You need the \u003cstrong\u003eright people\u003c/strong\u003e, the \u003cstrong\u003eright mindset\u003c/strong\u003e, and a clear \u003cstrong\u003emission\u003c/strong\u003e that everyone’s working towards. Otherwise, you’re just doing tasks without any thought behind it—and that’s a recipe for failure.\u003c/p\u003e\n\u003cp\u003eAnother thing: adapt your style of working to the needs of your business. there is a structure set by scrum you can adapt to. But dont be afraid to change some things if it works for the better for your company. The goal is creating a team that likes to work in the created structure and is able to deliver value instead of features.\u003c/p\u003e\n\u003cp\u003eYou as a team or even a Company need to \u003cstrong\u003emake it work\u003c/strong\u003e by putting the right people (with a vision and mandate) and processes in place, or you’re setting yourself up for a crash landing not delivering anything then frustration.\u003c/p\u003e\n\u003cp\u003eDont let solo managers trying to make themself look good take over your thrive for value and actually helping the business. Not hurting the business.\u003c/p\u003e\n","date_modified":"2024-09-17T09:14:28+02:00","date_published":"2024-09-10T22:03:00+02:00","id":"https://www.allsubjectsmatter.nl/scrum-and-the-army-a-different-kind-of-mission/","language":"en","summary":"Scrum isn’t a quick fix. It’s like planning a mission like the army. A sprint isn’t about speed, it’s about precision, teamwork, and adapting to the unknowns along the way.","tags":["Teams"],"title":"Scrum and the army, a different kind of mission.","url":"https://www.allsubjectsmatter.nl/scrum-and-the-army-a-different-kind-of-mission/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch3 id=\"introduction\"\u003eIntroduction\u003c/h3\u003e\n\u003cp\u003eThis post aims to shed light on database performance within Azure, focusing on a direct comparison between MySQL, MariaDB, and PostgreSQL. Despite abundant literature on the subject, practical tests that measure real-life performance are scarce.\u003c/p\u003e\n\u003cp\u003eMany discussions revolve around the differences between these databases rather than their actual performance. When I started this research, my priority wasn\u0026rsquo;t the differences but finding a performant database that could serve as a viable Azure-based alternative for our MariaDB Monolithic setup. Initially, we thought switching from MariaDB to MySQL was a straightforward replacement with almost no performance impact. We were wrong there!\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003e\u0026ldquo;There are many opinions about performance, But we needed facts\u0026rdquo;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eWe encountered a massive regression in performance in every query. Performance is crucial to us, and it was perplexing to see MySQL lag behind MariaDB so much. Upon discovering this, we set up a MariaDB VM ourselves in Azure, which proved to be performant again but introduced a lot of additional maintenance and management challenges, such as ensuring uptime, hot standby, and backups.\u003c/p\u003e\n\u003cp\u003eThis led me to explore PostgreSQL, hoping for better performance than what MySQL offered. With this introduction, let\u0026rsquo;s dive into the tests we conducted, including a complex query and a set of small queries, to test the performance differences between the three databases and their implementations within Azure. We also included Docker tests to be able to determine whether the performance discrepancies are database-related or if Azure is the contributing factor.\u003c/p\u003e\n\u003ch2 id=\"testing-our-complex-query\"\u003eTesting our complex Query\u003c/h2\u003e\n\u003cp\u003eTo assess the databases, we ran a complex query that aggregated data from multiple tables, representing a realistic workload for our system. For confidentiality reasons, we\u0026rsquo;ve altered table and column names in this description. This query aimed to evaluate how each database handles data aggregation and complex joins.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-sql\"\u003e\u003ccode class=\"language-sql\"\u003eWITH AggregatedSkateData AS (\nSELECT\n    e.id AS eventID,\n    e.theme_cid AS themeCID,\n    s.id AS skateID,\n    p.firstName || \u0026amp;#39; \u0026amp;#39; || p.lastName AS participantName,\n    l.label AS locationLabel,\n    s.priceOffered,\n    s.inspectionCompleted,\n    s.registrationDate,\n    s.eventStartDate,\n    org.name AS organizerName,\n    \u0026amp;#39;https://skateeventimages.net/\u0026amp;#39; || im.path AS organizerImageURL,\n    s.registrationCode,\n    COUNT(DISTINCT CASE WHEN pa_last.status_cid = 0 OR pa_last.id IS NULL THEN sp.id END) AS partsPending,\n    COUNT(DISTINCT CASE WHEN pa_last.status_cid = 1 THEN sp.id END) AS partsRequested,\n    COUNT(DISTINCT CASE WHEN pa_last.status_cid = 2 THEN sp.id END) AS partsConfirmed,\n    COUNT(DISTINCT CASE WHEN pa_last.status_cid = 8 THEN sp.id END) AS partsDenied,\n    MAX(CASE WHEN ti_unscanned.id IS NOT NULL THEN 1 ELSE 0 END) AS hasUnscannedTickets,\n    s.isPublic\nFROM\n    skateEvents s\n    INNER JOIN events e ON s.eventID = e.id\n    INNER JOIN participants p ON p.id = e.participantID\n    LEFT JOIN organizers org ON org.id = e.organizerID\n    LEFT JOIN images im ON im.id = org.profileImageID\n    LEFT JOIN locations l ON l.id = s.locationID\n    LEFT JOIN skateParts sp ON sp.eventID = s.id\n    LEFT JOIN partApprovals pa ON pa.id = sp.partApprovalID\n    LEFT JOIN ticketsUnscanned ti_unscanned ON ti_unscanned.id = sp.ticketID AND ti_unscanned.scannedAt IS NULL\n    LEFT JOIN partApprovalActions pa_last ON pa_last.id = (\n        SELECT pa2.id\n        FROM partApprovalActions pa2\n        WHERE pa2.partID = pa.id\n        ORDER BY pa2.id DESC\n        LIMIT 1\n    )\nWHERE\n    s.registrationDate IS NOT NULL\n    AND e.cancelledAt IS NULL\n    AND e.deletedAt IS NULL\n    AND s.isArchived = false\n    AND s.deletedAt IS NULL\nGROUP BY\n    e.id, e.theme_cid, s.id, p.id, p.firstName, p.lastName, org.id, im.id, l.id, l.label, s.priceOffered, s.inspectionCompleted, s.registrationDate, s.eventStartDate, org.name, im.path, s.registrationCode, s.isPublic\n)\nSELECT * FROM AggregatedSkateData\nWHERE\n    TO_CHAR(s.registrationDate, \u0026amp;#39;YYYY-MM\u0026amp;#39;) = \u0026amp;#39;2024-03\u0026amp;#39;\n    OR partsDenied \u0026#43; partsRequested \u0026#43; partsPending \u0026amp;gt; 0\nORDER BY s.registrationDate DESC\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch3 id=\"azure-complicated-query\"\u003eAzure, complicated query\u003c/h3\u003e\n\u003cp\u003eWe conducted these tests within the same Azure zone for containers and databases to avoid interzone latency. Below are the specifications and results for transparency. It\u0026rsquo;s worth noting that for MariaDB and MySQL, the performance specifications didn\u0026rsquo;t markedly affect outcomes, provided we avoided the first (burstable) tier. PostgreSQL was the exception, showing noticeable improvements in large query runtimes.\u003c/p\u003e\n\u003cp\u003eUsing HeidiSQL via the Azure VPN, we determined our best results out of five runs.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMySQL Flexible, Zone 2 (Standard_E4ds_v5, 450 iops)\u003c/strong\u003e\u003cbr\u003e\nFound rows: 130\u003cbr\u003e\nDuration for 1 query: 13,875 sec\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMariaDB VM, Zone 2 (Standard B4s v2 (4 vcpus, 16 GiB memory))\u003c/strong\u003e\u003cbr\u003e\nFound rows: 130\u003cbr\u003e\nDuration for 1 query: 10,625 sec\u003cbr\u003e\n23.4% better then MySQL\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePostgres Flexible, Zone 3 (Standard_E4ds_v50 p10, 500 iops)\u003c/strong\u003e\u003cbr\u003e\nFound rows: 130\u003cbr\u003e\nDuration for 1 query: 3,953 sec\u003cbr\u003e\n71.5% better then MySQL\u003c/p\u003e\n\u003ch3 id=\"local-docker-complicated-query\"\u003eLocal Docker, complicated query\u003c/h3\u003e\n\u003cp\u003eLocal hardware we tested on\u003c/p\u003e\n\u003cp\u003e12th Gen Intel(R) Core(TM) i9-12900H, 2900 Mhz, 14 Core(s), 20 Logical Processor(s)\u003cbr\u003e\nInstalled Physical Memory (RAM) 32,0 GB\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMySQL 8.3 Docker\u003c/strong\u003e\u003cbr\u003e\nFound rows: 130\u003cbr\u003e\nDuration for 1 query: 12,437 sec\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMariaDB 11.4 Docker\u003c/strong\u003e\u003cbr\u003e\nFound rows: 130\u003cbr\u003e\nDuration for 1 query: 9,484 sec\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePostgres 16.2 Docker\u003c/strong\u003e\u003cbr\u003e\nFound rows: 130\u003cbr\u003e\nDuration for 1 query: 3,969 sec\u003c/p\u003e\n\u003cp\u003eThis concludes our complicated query tests. It is awesome to see Postgres perform so well here! We used to run in MariaDB before we started our move to Azure. It was very sad to see such bad performance in MySQL the moment we started testing with this new setup.\u003c/p\u003e\n\u003ch2 id=\"testing-our-set-small-querys\"\u003eTesting our set small Query\u0026rsquo;s\u003c/h2\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/mysql-mariadb-postgresql-performance-compared-in-azure/images/image-2_hu_c11062909b9cd6ee.webp\" alt=\"\" width=\"560\" height=\"171\" data-zoom-src=\"https://www.allsubjectsmatter.nl/mysql-mariadb-postgresql-performance-compared-in-azure/images/image-2_hu_44990348cc6259d6.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003cp\u003eThe next test is when we fire more then 20.000 transactions per second against each database which will build us a feed in JSON that takes a certain amount of time to complete. we stress our database better with a lot of query\u0026rsquo;s which will give us a average runtime for every database.\u003cbr\u003e\nThere is also some PHP logic here since this is a PHP script we run to generate the feed. But it runs on the same hardware/docker image all the time. so the difference in runtime comes from the database.\u003cbr\u003e\nFor this we moved our containers to the same zone as our database is running to also take away potential issues with zone latency.\u003c/p\u003e\n\u003cp\u003eWe are running this script within our Laravel framework. We are running our PHP against \u003ca href=\"https://openswoole.com/\" data-external\u003eSwoole\u003c/a\u003e for the best possible performance within PHP 8.\u003c/p\u003e\n\u003ch3 id=\"azure-small-querys\"\u003eAzure, small query\u0026rsquo;s\u003c/h3\u003e\n\u003cp\u003eIn this test we also introduce container performance aspects .\u003cbr\u003e\nWe run our containers and database again in the same Zone in Azure (Western Europe). When using Azure we run with a lot less performance than we have available in our docker environment.\u003c/p\u003e\n\u003cp\u003eFor this we can only use these results to compare the differences within docker and Azure itself, not between them. Comparing the two together wont mean anything due to this hardware difference. We only want to observe the difference in runtime against the databases. and Docker is a baseline to check if the differences match up with the differences we see in Azure.\u003c/p\u003e\n\u003cp\u003eOur Node Size in Azure is a standard_b2s, 2 vCPUs with 4 GiB of RAM\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAPI, Zone 2 -\u0026gt; MariaDB VM, Zone 2\u003c/strong\u003e\u003cbr\u003e\n24.59s runtime (spot nodes)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAPI, Zone 2 -\u0026gt; MySQL Flexible, Zone 2\u003c/strong\u003e\u003cbr\u003e\n27.87s runtime (spot nodes)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAPI, Zone 3 -\u0026gt; Postgres Flexible, Zone 3\u003c/strong\u003e\u003cbr\u003e\n24.92s runtime (spot nodes)\u003c/p\u003e\n\u003ch3 id=\"local-docker-small-querys\"\u003eLocal Docker, small query\u0026rsquo;s\u003c/h3\u003e\n\u003cp\u003eFor the small query tests within local Docker we utilize a system powered by a 12th Gen Intel(R) Core(TM) i9-12900H CPU, running at 2900 MHz with 14 cores and 20 logical processors, complemented by 32 GB of installed RAM.\u003c/p\u003e\n\u003cp\u003eThese Docker images are configured to be memory-bound, meaning their performance was primarily dependent on the available memory rather than being limited by the CPU. The CPU settings were configured to allow unrestricted performance, ensuring that Docker can utilize as much CPU resource as needed.\u003c/p\u003e\n\u003cp\u003eBefore each test run against a Database, I made sure to restart the Docker container to refresh its connection. After restarting, I allowed a 30-second pause to ensure that the container had fully stabilized and was running idle. This pause is crucial for ensuring more consistent test conditions. Once the container was ready, I initiated the feed generator script to begin building the feed, assessing how effectively our different database setups could handle small queries under these controlled conditions.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAPI -\u0026gt; MariaDB 11.4 Docker\u003c/strong\u003e\u003cbr\u003e\n22.88s runtime\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAPI -\u0026gt; MySQL 8.3 Docker\u003c/strong\u003e\u003cbr\u003e\n23.74s runtime\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAPI -\u0026gt; Postgres 16.2 Docker\u003c/strong\u003e\u003cbr\u003e\n20.77s runtime\u003c/p\u003e\n\u003ch2 id=\"mariadb-query-cache\"\u003eMariaDB query cache\u003c/h2\u003e\n\u003cp\u003eIn our past use of MariaDB within our monolithic setup, we made significant use of the query cache functionality. This feature played a crucial role in delivering performance for repeated queries. This efficiency persists until the cache requires rebuilding, typically triggered by updates or inserts into the cached query\u0026rsquo;s tables. Lets do a second run on our complex query with Query cache enabled.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLocal MariaDB Docker\u003c/strong\u003e\u003cbr\u003e\nFound rows: 130\u003cbr\u003e\nDuration for 1 query: 0,000 sec\u003c/p\u003e\n\u003cp\u003eHowever, the performance benefits of the query cache aren\u0026rsquo;t universally consistent. For instance, when executing an average of 20,000 transactions per second with small queries, the runtime dropped to some extent. This variance highlights that the query cache doesn\u0026rsquo;t always produce the same level of optimization, particularly under high-load conditions. but more about that topic later.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAPI -\u0026gt; Local MariaDB Docker (query cache enabled)\u003c/strong\u003e\u003cbr\u003e\n17.30s runtime\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAPI, Zone 2 -\u0026gt; MariaDB VM, Zone 2 (query cache enabled)\u003c/strong\u003e\u003cbr\u003e\n22.72s runtime (spot nodes)\u003c/p\u003e\n\u003cp\u003eThe query cache in MariaDB works by storing the result set of SELECT queries, along with the query text. When an identical query is executed again (with the same text and under the same database user privileges), MariaDB can bypass the query execution phase, pulling the result directly from the cache. This feature can drastically reduce query execution times for frequently run queries that do not alter data.\u003c/p\u003e\n\u003cp\u003eIt\u0026rsquo;s important to approach the use of MariaDB\u0026rsquo;s query cache with caution. Despite its benefits, the query cache was deprecated and disabled by default in newer versions of MariaDB 10.1.7 (and MySQL as of version 5.7.20) due to concerns over its impact on performance in environments with high levels of concurrent transactions. Nonetheless, for specific scenarios, particularly those with a predominant read workload, the query cache can still significantly enhance performance. It is completely removed in MySQL 8 and up.\u003c/p\u003e\n\u003ch2 id=\"observation-on-azures-flexible-servers\"\u003eObservation on Azure\u0026rsquo;s Flexible Servers\u003c/h2\u003e\n\u003cp\u003eThese tests highlight that the performance of databases on Azure\u0026rsquo;s flexible servers has improved significantly, aligning closely with the performance seen in local Docker environments. This marks a decent improvement from the previous Azure database single servers, which typically exhibited lesser performance due to some form of overhead introduced by how it was it was running in the Azure cloud.\u003c/p\u003e\n\u003ch2 id=\"azure-latency-anomaly\"\u003eAzure Latency anomaly\u003c/h2\u003e\n\u003cp\u003eAs an added bonus, we conducted some additional tests to measure interzone latency when connecting from Zone 1 or 2 to a PostgreSQL database in Zone 3 within Azure.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAPI, Zone 2 -\u0026gt; Azure PostgreSQL Flexible, Zone 3\u003c/strong\u003e\u003cbr\u003e\nThe runtime was 44.23 seconds.\u003c/p\u003e\n\u003cp\u003eInterestingly, the runtime doubled again, a phenomenon observed exclusively when communications occur between Zones 1-2 and Zone 3, or vice versa. This pattern suggests that any container requiring a database connection experiences significant latency under these conditions, whereas latency is nearly negligible when the interaction is between Zone 1 and Zone 2.\u003c/p\u003e\n\u003cp\u003eFor those who might have missed it, I\u0026rsquo;ve previously discussed a latency anomaly encountered in Zone 3. The conventional wisdom suggests maintaining the database connection within the same zone to minimize latency. However, our tests show that the latency impact is minimal when operating between Zones 1 and 2, but it becomes significantly pronounced when crossing from Zones 1 or 2 to Zone 3.\u003c/p\u003e\n\u003cp\u003eThis testing further confirms that the latency issue is not exclusive to MySQL Flexible services but also affects PostgreSQL Flexible services within Azure at least, indicating a broader pattern across Azure\u0026rsquo;s database offerings.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/azure-western-europe-zone-3-latency-issue-exploration/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eLatency Anomaly: An attempt to untangle Azure Performance issues in Western Europe\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eUnveiling a significant Azure performance issue: Zone 3’s unexpected latency anomaly in the Western Europe region challenges cloud deployment strategies\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/mysql-mariadb-postgresql-performance-compared-in-azure/bookmarks/01-icon_hu_9b466dd874965fcb.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco Loup.\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/mysql-mariadb-postgresql-performance-compared-in-azure/bookmarks/01-thumb_hu_27e6839ff9d3a962.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eFrom a performance related evaluation, it\u0026rsquo;s clear that choosing the right database is crucial and should match your app\u0026rsquo;s specific needs for speed and efficiency. The results show MySQL tends to be slower. MariaDB works well and has a handy query cache feature, but Azure has no support for MariaDB anymore. This makes PostgreSQL the most viable choice for performance-focused users in Azure, as it excels in handling both complex and numerous simple queries effectively as shown in the tests above.\u003c/p\u003e\n\u003cp\u003eThis analysis hopefully helps in selecting the right database technology and underlines how important careful planning is for boosting your app performance. For those using Azure and looking for top tier performance, PostgreSQL is the go-to option in my test results.\u003c/p\u003e\n\u003cp\u003eThere are other reasons to choose your database on. But i only wanted to focus on the performance aspect in this post. This was our main concern since we are in the proces of moving from a Monolithic server to Azure. There are more things to consider when switching databases. The advantage here is that all three databases can be integrated quite easily if you use either of them.\u003c/p\u003e\n\u003cp\u003ePostgres is the best compatible here with its strict routines. The complicated query above is written for Postgres and runs in MariaDB without a single change. For MySQL the TO_CHAR in the last line only needs to be replaced with DATE_FORMAT.\u003c/p\u003e\n","date_modified":"2024-03-30T18:59:39+01:00","date_published":"2024-03-29T00:04:45+01:00","id":"https://www.allsubjectsmatter.nl/mysql-mariadb-postgresql-performance-compared-in-azure/","language":"en","summary":"Exploring Azure's database performance while on a journey through MySQL, MariaDB, and PostgreSQL. Find out which database leads the race.","tags":["Under the hood"],"title":"Azure + MySQL, MariaDB, PostgreSQL performance compared","url":"https://www.allsubjectsmatter.nl/mysql-mariadb-postgresql-performance-compared-in-azure/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eAs we embark on a transition from a traditional VM setup hosted at TransIP to the scalable cloud infrastructure of Microsoft Azure, our journey is as much about discovery as it is about performance. Guided by the expertise of \u003ca href=\"https://www.pionative.com/\" data-external\u003ePionative\u003c/a\u003e, we\u0026rsquo;re not just moving our environment; we are attempting to reinvent it in Azure, ensuring that every step we take is carefully tested and analyzed. Our goal? To ensure optimal performance in this new architecture and to uncover any potential bottlenecks or hidden issues that might lurk within our new cloud architecture as early as possible.\u003c/p\u003e\n\u003ch2 id=\"azure-regions-and-zones\"\u003eAzure regions and Zones\u003c/h2\u003e\n\u003cp\u003eAzure\u0026rsquo;s infrastructure spans the globe, organized into regions for localized services, and further divided into zones for high availability. \u003cstrong\u003eRegions\u003c/strong\u003e are specific geographical areas hosting Azure services, while \u003cstrong\u003ezones\u003c/strong\u003e are physical locations within these regions, each with independent utilities to isolate from failures.\u003c/p\u003e\n\u003ch2 id=\"the-western-europe-region\"\u003eThe Western Europe Region\u003c/h2\u003e\n\u003cp\u003eThe Western Europe region in Azure is structured into three distinct availability zones. These zones provide an additional layer of redundancy and fault tolerance by physically separating data centers within the region. This setup is designed to ensure that even in the event of a complete zone failure, the services can continue to operate unaffected in the other zones, offering robust disaster recovery solutions for businesses operating in Europe. Where Zone 1,2 and 3 are exactly located geographically is unknown to me.\u003c/p\u003e\n\u003ch2 id=\"performance-anomaly-in-the-western-europe-region\"\u003ePerformance anomaly in the Western Europe Region\u003c/h2\u003e\n\u003cp\u003eOur testing regime led us to a anomaly that has become the focal point of our Azure exploration for this post: a distinct latency issue within Zone 3 of the Western Europe region. This discovery wasn\u0026rsquo;t immediate; it emerged from days of performance testing and a keen observation of patterns that deviated from our expectations. This all because we just started to learn the way’s of Azure. And it’s difference in performance as opposed to our current setup.\u003c/p\u003e\n\u003cp\u003eWe first started to exclude our own pods, code, performance settings within the node pool. Database performance settings and such.\u003c/p\u003e\n\u003ch2 id=\"methodology-and-findings\"\u003eMethodology and findings\u003c/h2\u003e\n\u003cp\u003eOur final approach was quite thorough to verify this result. We deployed two node pools across each of Microsoft Azure\u0026rsquo;s three zones within the Western Europe region, aiming to rule out any node-specific variances. All these pools had the exact same performance setup.\u003c/p\u003e\n\u003cp\u003eThrough extensive testing, including a battery of database calls and running our cron jobs integral to our application\u0026rsquo;s functionality, we noticed a consistent and surprising pattern. Irrespective of the zone\u0026rsquo;s reported capacity or congestion levels.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eInteractions with the database were significantly slower when either the database or the connecting pods were situated in Zone 3.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"performance-results\"\u003ePerformance results\u003c/h2\u003e\n\u003cp\u003eBelow is our situation running a Cronjob that takes us locally within Docker around 32 seconds to run against our current setup, which isn’t comparable to MS Azure. But it did provide us a performance Matrix and goal. The script (cronjob) we ran fires around 32,000 queries to build a feed from all the products on our website. Yes, that’s a lot, but that is not the matter in this case. We wanted to uncover the performance issue. We know this script can be optimized to perform better.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMySQL Database in Zone 2\u003c/strong\u003e\u003cbr\u003e\nZone 1 – pool 1: 33 seconds\u003cbr\u003e\nZone 1 – pool 2: 32 seconds\u003c/p\u003e\n\u003cp\u003eZone 2 –  pool 1:  32 seconds\u003cbr\u003e\nZone 2 –  pool 2:  32 seconds\u003c/p\u003e\n\u003cp\u003eZone 3 –  pool 1: 115 seconds\u003cbr\u003e\nZone 3 –  pool 2: 117 seconds\u003c/p\u003e\n\u003cp\u003eThen, to put the icing on the cake, we switched the setup with a database running in Zone 3. And see below the expected results we hoped to see to conclude this search.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMySQL Database on Zone 3\u003c/strong\u003e\u003cbr\u003e\nZone 1: 120 seconds\u003cbr\u003e\nZone 2: 122 seconds\u003cbr\u003e\nZone 3: 33 seconds\u003c/p\u003e\n\u003cp\u003eWe started to see the same consistency every time, even when we ran less but larger database calls.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMySQL Database on zone 2\u003c/strong\u003e\u003cbr\u003e\n16 DB queries single threaded.\u003c/p\u003e\n\u003cp\u003eZone 1: Total time for queries from log: 9.57 ms\u003cbr\u003e\nZone 2: Total time for queries from log: 9 ms\u003cbr\u003e\nZone 3: Total time for queries from log: 49.96 ms\u003c/p\u003e\n\u003ch2 id=\"website-performance-in-azure\"\u003eWebsite performance in Azure\u003c/h2\u003e\n\u003cp\u003eWhen we run our Database in Zone 2 while having the API pod in Zone 1, the initial load time for our website decreases by at least 400 milliseconds for the first page load, compared to the configuration where the API pod is in Zone 3.\u003c/p\u003e\n\u003cp\u003eThis means that the setup with the database in Zone 2 and the API pod in Zone 1 results in a much faster initial response time for the website compared to having the API pod located in Zone 3 and the Database in Zone 2.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/azure-western-europe-zone-3-latency-issue-exploration/images/image-1_hu_239c52cf6c075a2e.webp\" alt=\"\" width=\"560\" height=\"129\" data-zoom-src=\"https://www.allsubjectsmatter.nl/azure-western-europe-zone-3-latency-issue-exploration/images/image-1_hu_5c52b5671bc6fe37.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eThe initial first call performance of our website when run in an optimal setup in Azure.\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003cp\u003eThere is a margin of error here, but we can conclude from these results that when you want to run your API demographically separated in two zones, there is no way you can consistently perform when using Zone 3. Zones 1 and 2 are the way to go.\u003c/p\u003e\n\u003cp\u003eWe must be honest that this performance anomaly only happens when you have a pod with a database connection. Running our website (which has no database connection) in Zone 3 measured almost no impact here when the API pod ran in Zone 1 and the database resided in Zone 2.\u003c/p\u003e\n\u003ch2 id=\"diving-deeper-in-zone-3s-latency-issues\"\u003eDiving deeper in Zone 3\u0026rsquo;s latency issues.\u003c/h2\u003e\n\u003cp\u003eThis performance discrepancy in Zone 3 was astonishing to us. According to Microsoft\u0026rsquo;s best practices, the physical proximity of application servers to their respective databases within the same region should minimize latency. Our findings, however, painted a different picture, suggesting that within-region zone selection could profoundly impact application performance.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/azure-western-europe-zone-3-latency-issue-exploration/images/data-src-image-68bb083d-b85e-44ae-9919-8404530f242e_hu_f23e2d3a977b47e3.webp\" alt=\"\" width=\"560\" height=\"235\" data-zoom-src=\"https://www.allsubjectsmatter.nl/azure-western-europe-zone-3-latency-issue-exploration/images/data-src-image-68bb083d-b85e-44ae-9919-8404530f242e_hu_dda4b94313ff73ba.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003cp\u003eOur investigation was quite comprehensive, eliminating potential variables such as our own codebase, PHP environment configurations, MySQL performance, and much more. The consistent latency issues, particularly affecting PHP\u0026rsquo;s single-threaded cron jobs, finally pointed us towards a more systemic issue within Zone 3\u0026rsquo;s network architecture or deployment somewhere.\u003c/p\u003e\n\u003ch2 id=\"collaboration-and-learning\"\u003eCollaboration and learning\u003c/h2\u003e\n\u003cp\u003eOur journey, backed by the collaboration with \u003ca href=\"https://www.pionative.com/\" data-external\u003ePionative\u003c/a\u003e, underscores the importance of partnership in navigating the cloud\u0026rsquo;s complexity. As we dove deeper into this latency anomaly, our shared efforts aim not only to optimize our deployment but also to contribute our findings back to a broader Azure community.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eThis latency impact is too large to ignore in our production environment.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"implications-and-next-steps\"\u003eImplications and next steps\u003c/h2\u003e\n\u003cp\u003eThis anomaly in Zone 3 opens up discussions on optimal zone selection and deployment strategies within Azure\u0026rsquo;s Western Europe region. Is this the reason that Zone 3 is the least populated zone? Online there isn’t much to find about this it looks like. So i wrote this post since this cost us literally days to pinpoint this performance difference.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThis will impact our production environment. We are going to run our DB in Zone 1 or 2. and the API must reside in either of these zones. Zone 3 can be used for pods that talk with the API.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThis latency anomaly in Zone 3 was more then a challenge for us. We took it as a opportunity to learn, adapt, and optimize our knowledge. As we continue to collaborate, test, and refine our approach, our journey into Azure is shaping up to be a very interesting technical exploration which is broadening our toolset for the future.\u003c/p\u003e\n","date_modified":"2024-03-05T17:28:41+01:00","date_published":"2024-03-05T12:16:52+01:00","id":"https://www.allsubjectsmatter.nl/azure-western-europe-zone-3-latency-issue-exploration/","language":"en","summary":"Unveiling a significant Azure performance issue: Zone 3's unexpected latency anomaly in the Western Europe region challenges cloud deployment strategies","tags":["Under the hood"],"title":"Latency Anomaly: An attempt to untangle Azure Performance issues in Western Europe","url":"https://www.allsubjectsmatter.nl/azure-western-europe-zone-3-latency-issue-exploration/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eThis post is a follow up on GIT deployment strategies within Kubernetes environments. Make sure you read this post first:\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/ci-cd-pipeline-with-docker-and-git-branching-strategy/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eCI/CD pipeline with Docker and Git branching strategy\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eExplore advanced CI/CD with Docker \u0026amp; Git. Master deploying Docker artifacts via Git branching. Dive into strategies for DEV, STAGING, LIVE environments. Perfect your pipeline now!\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/hotfixing-with-docker-artifacts-in-a-single-branch-strategy/bookmarks/01-icon_hu_9b466dd874965fcb.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco Loup.\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/hotfixing-with-docker-artifacts-in-a-single-branch-strategy/bookmarks/01-thumb_hu_4b0a9042ec0e16fc.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cp\u003eIn a CI/CD pipeline centered around Docker artifacts with the focus on a singular branch, implementing an efficient hotfix strategy requires a small deviation to ensure rapid deployment of critical fixes. Let\u0026rsquo;s work out our approach to hotfixing in this Docker-centric environment, given the explained branching architecture in the above post.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/hotfixing-with-docker-artifacts-in-a-single-branch-strategy/images/Git-branches-and-docker-environments-ci_cd-1_hu_17e1895a357c0fa7.webp\" alt=\"\" width=\"560\" height=\"378\" data-zoom-src=\"https://www.allsubjectsmatter.nl/hotfixing-with-docker-artifacts-in-a-single-branch-strategy/images/Git-branches-and-docker-environments-ci_cd-1_hu_7bd9f2a51f818afa.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eGraphical reference of our branching and deployment strategy\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"identifying-the-need-for-a-hotfix\"\u003eIdentifying the need for a Hotfix\u003c/h2\u003e\n\u003cp\u003eHotfixes are a critical part of maintaining the stability and reliability of any software in production. They are typically required under specific circumstances that demand immediate attention. Understanding when to implement a hotfix is key to managing your CI/CD pipeline effectively, especially when using Docker artifacts and streamlined branching strategies like focusing primarily on a Single branch. Here are our scenarios where a hotfix can be necessary:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eSecurity Vulnerabilities:\u003c/strong\u003e One of the most urgent triggers for a hotfix is the discovery of a security vulnerability within your application. Whether it\u0026rsquo;s an exposed data risk, a potential for unauthorized access, or any flaw that could be exploited maliciously, quick action is required to patch the vulnerability and protect your users and data.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eCritical Bugs Impacting User Experience:\u003c/strong\u003e If a significant bug slips through your testing processes and reaches production, it can severely impact the user experience. This could be anything from a broken core functionality that prevents users from using your application as intended to severe performance issues leading to timeouts or crashes. If the issue significantly affects your user base, a hotfix is warranted.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eCompliance Issues:\u003c/strong\u003e Certain applications may have legal or regulatory requirements. If a compliance issue is identified that could affect your legal standing or user privacy, addressing it swiftly through a hotfix becomes a priority to avoid penalties or breaches of trust.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eData Loss Risks:\u003c/strong\u003e Scenarios where there\u0026rsquo;s a risk of data loss or corruption for users require immediate attention. Whether it\u0026rsquo;s a flaw in how data is saved, updated, or retrieved, a hotfix should be deployed to mitigate any potential loss or corruption of user data.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003ePayment or Transaction Failures:\u003c/strong\u003e For applications dealing with payments or any form of transactions, issues that prevent users from completing transactions or that might lead to financial discrepancies are critical. Such problems not only affect user trust but can also have financial repercussions for both users and the service provider, necessitating a hotfix.\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"deploying-a-hotfix\"\u003eDeploying a Hotfix\u003c/h2\u003e\n\u003cp\u003eWhen a critical issue is identified within the production environment, the immediate goal is to develop, test, and deploy a hotfix as quickly as possible. Given that our primary branch for development is MAIN and that we generate Docker artifacts for deployment, we need a process that allows for rapid intervention without disrupting the ongoing work.\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eCreating a Hotfix Branch\u003c/strong\u003e: Instead of branching off from your Production branch (as LIVE is not represented anymore in the Git repository in this strategy), the hotfix branch is created from the commit currently deployed in production. This ensures that the Hotfix is based on the exact state of the version running on the production environment. The specific commit can be identified through tags or Docker image labels that correlate with the production deployment.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDeveloping and Testing the Hotfix\u003c/strong\u003e: The critical fix is developed within this hotfix branch. The urgency of the situation necessitates a focused approach, targeting the problem with minimal changes to expedite the testing and deployment process. Despite the rush, adherence to the Definition of Done (DoD) is crucial to ensure the stability and reliability of the hotfix. This can mean a condensed version of the DoD, prioritizing critical tests and checks that ensure the fix will not introduce new issues.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eCreating a Docker Artifact for the Hotfix\u003c/strong\u003e: Once the hotfix meets the agreed DoD criteria, an automated process builds a Docker image from the hotfix branch. This Docker artifact is then tagged appropriately, indicating its purpose and urgency.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDeploying the Hotfix\u003c/strong\u003e: The newly created Docker artifact is deployed directly into the production environment, bypassing the usual DEV to STAGING to PRODUCTION environment strategy. This deployment is only justified by the critical nature of the fix and the testing it has undergone.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eMerging Back and Maintaining Consistency\u003c/strong\u003e: After the hotfix is successfully running in production, it\u0026rsquo;s essential to merge the changes back into the MAIN branch and any other relevant branches or tags. Only this step ensures that the fix is incorporated into the ongoing development work and future releases, preventing regression!!\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"ensuring-rapid-deployment-and-minimizing-disruption\"\u003eEnsuring Rapid Deployment and Minimizing Disruption\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eAutomated Processes\u003c/strong\u003e: Automation plays a key role in this strategy, from the creation of the Docker artifact to its deployment. Automated tests, builds, and deployments ensure that the hotfix process is as fast and efficient as possible.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eCommunication and Monitoring\u003c/strong\u003e: Rapid deployment of hotfixes requires clear communication among team members and close monitoring of the production environment post-deployment to ensure the fix is effective and does not introduce new issues.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDocumentation and Review\u003c/strong\u003e: Documenting the hotfix process and conducting post-mortem reviews help in refining the hotfix strategy, ensuring that the team can respond even more effectively to future issues.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eIn a CI/CD pipeline using Docker and primarily operating through Git Flow, hotfixing requires a streamlined approach that leverages automation and focuses on rapid, targeted fixes. By creating hotfix branches based on the production-deployed commit, developing and testing fixes according to an well defined DoD, and deploying Docker artifacts directly into production, teams can address critical issues swiftly while ensuring the ongoing stability and integrity of their application.\u003c/p\u003e\n","date_modified":"2024-02-20T12:32:34+01:00","date_published":"2024-02-20T09:53:55+01:00","id":"https://www.allsubjectsmatter.nl/hotfixing-with-docker-artifacts-in-a-single-branch-strategy/","language":"en","summary":"Master hotfixes in Kubernetes with Docker. Learn to rapidly deploy critical fixes in a CI/CD pipeline focusing on streamlined branching. Perfect for tackling urgent issues without disrupting flow. Dive in now.","tags":["Teams","Under the hood"],"title":"Hotfixing with Docker Artifacts in a Git Flow Strategy","url":"https://www.allsubjectsmatter.nl/hotfixing-with-docker-artifacts-in-a-single-branch-strategy/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eLet\u0026rsquo;s dive into managing a CI/CD pipeline using Docker with a focus on Git branching strategy that involves development (DEV), staging (STAGING), and production (LIVE) branches. Our goal is to clarify the role each branch plays in the process, particularly how Docker artifacts are created and managed within this framework.\u003c/p\u003e\n\u003ch2 id=\"docker-deployment-using-git-flow-with-multiple-branches\"\u003eDocker deployment using Git Flow with multiple branches\u003c/h2\u003e\n\u003cp\u003eThe core of our branching strategy involved three branches:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eDEV\u003c/strong\u003e: This is where all development work happens. Every new feature, bug fix, or enhancement starts in a feature branch, branched off from DEV. Once the work on a feature branch is complete, reviewed and meets the Definition of Done (DoD), it\u0026rsquo;s merged back into DEV.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSTAGING\u003c/strong\u003e: This branch mirrors a pre-production environment. It\u0026rsquo;s where integrated features from DEV get tested as a whole and accepted by stakeholders to ensure they work together seamlessly.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eMASTER\u003c/strong\u003e: The branch that reflects the current state of our live environment. Only thoroughly tested and stakeholder-approved changes from STAGING make it to Production.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://dev.to/preethamsathyamurthy/git-branching-and-branching-strategy-4mci\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eGit Branching and Branching Strategy\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eIn this post, we will go over why branching is required, the difference between development, staging…\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/ci-cd-pipeline-with-docker-and-git-branching-strategy/bookmarks/01-icon_hu_4e78872e34382e92.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eDEV Community\u003c/span\u003e\u003cspan\u003ePreetham\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cp\u003eThis was our variation of a Git Flow strategy, we used until now. When you start working with Artifacts this all changes back to a normal Git Flow structure. We still have environments running (Dev, Staging and Prod). The real difference is that code commits get done once instead of getting merged into a branch running on Staging or in Production.\u003c/p\u003e\n\u003cp\u003eInstead we now have one Branch we called it Main. This is where a runnable artifact gets created after a successful code merge using Docker images. Everything deployed into our Staging or Production environment is a specific version of an earlier created Artifact.\u003c/p\u003e\n\u003cp\u003eThe biggest change is that STAGING and PROD aren’t branches anymore where you merge code into. They have instead became environments that run a specific version of a Docker image. This all has its own challenges, especially when you need to Hotfix. Hotfixing has it’s own post which you can read after this post. Lets break down all of this without introducing hotfixing step by step first. It is comprehensive enough already.\u003c/p\u003e\n\u003ch2 id=\"what-is-an-artifact\"\u003eWhat is an Artifact?\u003c/h2\u003e\n\u003cp\u003eA Docker artifact, often referred to interchangeably with a Docker image, is a lightweight, standalone, executable package that includes everything needed to run a piece of software, including the code, a runtime, libraries, environment variables, and config files. In simple terms, it\u0026rsquo;s like a snapshot of your application and its environment, bundled together, ready to be deployed and run anywhere Docker is installed, ensuring consistency across different environments.\u003c/p\u003e\n\u003ch2 id=\"the-importance-of-dod-when-merging-code\"\u003eThe Importance of DoD When Merging code\u003c/h2\u003e\n\u003cp\u003eA comprehensive and clear Definition of Done (DoD) plays a pivotal role in this new process, even more when merging changes into a single branch that triggers a Docker image which should be runnable and deployable:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eQuality Assurance\u003c/strong\u003e: The DoD ensures that all changes, including new features, enhancements, and hotfixes merged back into MAIN, meet a consistent quality standard. This includes code quality checks, successful completion of automated tests, and adherence to security best practices.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003ePreventing Regression\u003c/strong\u003e: By applying the DoD to all merges into MAIN, the team can prevent regressions and ensure that new changes do not negatively affect existing functionality. This is crucial for maintaining the integrity and stability of the codebase.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eStreamlining Deployments\u003c/strong\u003e: A clear and comprehensive DoD facilitates smoother deployments by ensuring that only fully vetted and ready-to-deploy changes are merged into MAIN. This reduces the risk of wasting resources on building Docker images that are bugged or broken, and therefore contributes to more efficient and reliable artifacts.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/ci-cd-pipeline-with-docker-and-git-branching-strategy/images/Git-branches-and-docker-environments-ci_cd_hu_17e1895a357c0fa7.webp\" alt=\"\" width=\"560\" height=\"378\" data-zoom-src=\"https://www.allsubjectsmatter.nl/ci-cd-pipeline-with-docker-and-git-branching-strategy/images/Git-branches-and-docker-environments-ci_cd_hu_7bd9f2a51f818afa.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eHere a graphical presentation of our Git Flow strategy and how this flows trough the different environments we have.\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"creating-and-managing-artifacts\"\u003eCreating and Managing Artifacts\u003c/h2\u003e\n\u003cp\u003eThe first thing for this proces to work is when a forked branch is merged back into MAIN, this triggers an automated process in our CI/CD pipeline. Here\u0026rsquo;s what happens:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eAutomated Testing\u003c/strong\u003e: First, automated tests run to ensure the new changes meet our quality standards. This is crucial for maintaining a stable branch.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDocker Artifact Creation\u003c/strong\u003e: Upon successful running al tests, a Docker image (our Artifact) is created from the MAIN branch. This is because the MAIN branch is the single branch that represents the latest state of every DONE development, This is where we generate our artifacts from that can become “shippable” products.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eWhy using MAIN for Docker Artifacts?\u003c/strong\u003e: Creating artifacts at this stage allows us to catch any integration issue. Since every feature merged into MAIN is going to be considered ready for a release, generating an artifact here ensures that we always have a tested and potentially deployable version of our product at hand.\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"transferring-from-branches-to-environments\"\u003eTransferring from Branches to Environments\u003c/h2\u003e\n\u003cp\u003eIn traditional Git workflows, branches like STAGING and PROD are used to represent different stages of the development process, from testing to production. However, in a Docker-centric CI/CD pipeline, the focus shifts from using these branches for code integration to using them as references for deployment environments.\u003c/p\u003e\n\u003ch2 id=\"deploying-to-staging\"\u003eDeploying to STAGING\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eCreating Docker Artifacts\u003c/strong\u003e: Once development is complete and merged into the MAIN branch, a Docker artifact is automatically created. This artifact is a snapshot of your application at that point in time, encapsulating all the changes that were merged.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDeployment, Not Merging\u003c/strong\u003e: The STAGING environment\u0026rsquo;s role is to test this Docker artifact in conditions that closely mimic the production environment. Instead of merging code from MAIN to a STAGING branch, you deploy the Docker artifact to a STAGING environment. This means the code itself isn\u0026rsquo;t moving between branches; rather, the compiled and containerized version of your application is being tested.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003ePurpose of STAGING\u003c/strong\u003e: The STAGING environment is essential for comprehensive testing, including performance and security assessments. It\u0026rsquo;s a final check to ensure everything works as expected in a production-like setting without directly affecting your actual users.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"going-to-production\"\u003eGoing to Production\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eStakeholder Approval\u003c/strong\u003e: Once your Docker artifact has been rigorously tested in STAGING and approved by stakeholders, it\u0026rsquo;s ready to move to production. This step is crucial for ensuring that all features, fixes, and updates meet the necessary standards and expectations.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDeployment to Production\u003c/strong\u003e: The transition from STAGING to PROD involves deploying the tested Docker artifact to your production environment. This deployment process ensures consistency, as the exact artifact tested in STAGING is what gets deployed to PROD, eliminating any discrepancies between tested and deployed code.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"key-takeaways\"\u003eKey takeaways\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eNo More Branch Merging for Deployment\u003c/strong\u003e: In this streamlined workflow, STAGING and PROD branches are no longer used for merging code. Instead, they reference specific deployment environments where Docker artifacts are deployed for testing and production.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eFocus on Artifacts\u003c/strong\u003e: The emphasis is on creating, testing, and deploying Docker artifacts. These artifacts represent stable versions of your application ready for STAGING and, eventually, PROD deployment.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eConsistency and Reliability\u003c/strong\u003e: By deploying the same Docker artifact from STAGING to PROD, you ensure that what has been tested is exactly what gets released, increasing the reliability and predictability of your deployments.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThis approach simplifies the deployment process, reduces the risk of errors, and ensures that every release is thoroughly tested in an environment that mirrors production as closely as possible.\u003c/p\u003e\n\u003ch2 id=\"are-multiple-branches-still-needed\"\u003eAre multiple Branches still needed?\u003c/h2\u003e\n\u003cp\u003eGiven this new workflow, the STAGING and PROD branches in Git branching serve no active purpose anymore and therefore they have became obsolete. They helped track which versions of the code have been deployed to each environment which we have solved by running a specific version of an Artifact.\u003c/p\u003e\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eIncorporating Docker into our Git branching strategy streamlines the development-to-deployment process by ensuring that every merge into MAIN triggers a Docker image with the potential to go to production. The key to this strategy is the creation of Docker artifacts early in the development cycle, allowing for continuous integration and delivery while maintaining high standards of quality and stability.\u003c/p\u003e\n\u003cp\u003eBy understanding the changes to Git branches and the process of creating and deploying Docker artifacts, we can rely on the complexities of CI/CD with confidence, ensuring that all new features and fixes are delivered to users as efficiently and reliably as possible.\u003c/p\u003e\n\u003ch2 id=\"hotfixing\"\u003eHotfixing\u003c/h2\u003e\n\u003cp\u003eWe havent spoken about hotfixing yet. As i said this deserves its own Post. You can read all about Hotfixing and its caveats here.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/hotfixing-with-docker-artifacts-in-a-single-branch-strategy/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eHotfixing with Docker Artifacts in a Single Branch Strategy\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eMaster hotfixes in Kubernetes with Docker. Learn to rapidly deploy critical fixes in a CI/CD pipeline focusing on streamlined branching. Perfect for tackling urgent issues without disrupting flow. Dive in now.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/ci-cd-pipeline-with-docker-and-git-branching-strategy/bookmarks/02-icon_hu_9b466dd874965fcb.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco Loup.\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/ci-cd-pipeline-with-docker-and-git-branching-strategy/bookmarks/02-thumb_hu_31f1067c4e9c7ce6.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n","date_modified":"2024-02-20T12:30:01+01:00","date_published":"2024-02-20T09:50:55+01:00","id":"https://www.allsubjectsmatter.nl/ci-cd-pipeline-with-docker-and-git-branching-strategy/","language":"en","summary":"Exploring CI/CD with Docker \u0026 Git. Mastering deploying Docker artifacts via Git branching. Dive into strategies for DEV, STAGING, PROD environments. Perfect your pipeline!","tags":["Teams","Under the hood"],"title":"CI/CD pipeline with Docker and Git branching strategy","url":"https://www.allsubjectsmatter.nl/ci-cd-pipeline-with-docker-and-git-branching-strategy/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eAs someone interested in the ever-evolving world of technology, I’ve witnessed a lot of developments in AI, particularly in the realms of language processing and user interaction. Reflecting on these advancements, I\u0026rsquo;ve been wanting to share why, in my opinion, the recent strides in AI aren\u0026rsquo;t as surprising as you might find them to be.\u003c/p\u003e\n\u003cp\u003eI use different pronounces as i refer to the same thing. I use AI and LLM (Large Language Model) for todays take on systems like ChatGPT.\u003c/p\u003e\n\u003ch2 id=\"my-first-encounters-with-ai\"\u003eMy first Encounters with \u0026ldquo;AI\u0026rdquo;\u003c/h2\u003e\n\u003cp\u003eMy journey with LLMs began earlier than some people realize. I had my first experiences with Flow.ai around 2018/2019. Back then, it was clear to me that we were on the brink of something. However, the general public seemed largely unaware of these undercurrents in AI development. The fact is, companies like Flow.ai (now part of Khoros) were laying the foundation for AI systems we see today like ChatGPT.\u003c/p\u003e\n\u003cp\u003eRead up on a post i wrote back in 2019 about our experiences with Flow.AI together with Stefan Samba.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/how-333travel-is-revolutionizing-the-travel-industry/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003e2019 - Blast from the past: How 333travel is revolutionizing the Travel industry\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eDiscover how 333travel’s AI ‘Joy’ redefines customer interaction, blending AI’s intent understanding with personal touch. A game-changing journey in AI-led travel solutions!\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/why-ai-caught-you-by-surprise/bookmarks/01-icon_hu_9b466dd874965fcb.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco Loup.\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/why-ai-caught-you-by-surprise/bookmarks/01-thumb_hu_e0b56fb8f4d4ac9.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"flowai-a-missed-opportunity-for-333travel\"\u003eFlow.ai: A Missed Opportunity for 333travel\u003c/h2\u003e\n\u003cp\u003eMy experience with Flow.ai was an eye-opener back then. It was a game-changer in the AI space for me, demonstrating the potential of neural network-based conversational AI. Looking back at it, when we tried integrating this technology at 333travel back in 2018, we might have been to far ahead of our time. Our natural style of helping people with asking for a proposal, did not gain the traction we hoped for using a LLM. Our customers (luckily?) preferred interacting with human specialists rather than bots. This was a clear indication that while the technology was promising, market readiness was another story.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003ePeople hated talking to the \u0026ldquo;oldstyle\u0026rdquo; chatbots not understanding what you meant due to the lack of intent understanding.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"the-siri-example\"\u003eThe Siri example\u003c/h2\u003e\n\u003cp\u003eLet\u0026rsquo;s talk a bit about Siri, Apple\u0026rsquo;s virtual assistant. Since its launch in 2011, Siri has made some strides. However, from my point of view, these improvements have been modest at best in the intent learning space. Siri, along with other tools like Cortana and Google Assistant, still required very specific commands and often struggled with understanding intent. While there has been some progress there, it\u0026rsquo;s been to slow of a curve in understanding human language and mainly intent.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003e2010-2011\u003c/strong\u003e: Siri\u0026rsquo;s initial years saw its integration into the iPhone 4S, offering basic voice recognition and command execution.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003e2012-2013\u003c/strong\u003e: Expansion into new languages and dialects, broadening its user base.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003e2014-2015\u003c/strong\u003e: Entry into new markets and the launch of SiriKit, allowing third-party app integration.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003e2016-2018\u003c/strong\u003e: Introduction to macOS and the release of Shortcuts for customized voice commands.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003e2020-2021\u003c/strong\u003e: moderate improvements in language translation, integration with Apple\u0026rsquo;s ecosystem, and enhanced speech recognition.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"ai-and-customer-interaction\"\u003eAI and Customer Interaction\u003c/h2\u003e\n\u003cp\u003eAs a technology enthusiast, I find ChatGPT impressive in many ways, but it\u0026rsquo;s essential to acknowledge that it\u0026rsquo;s no substitute for the detailed understanding that human specialists provide. Lets take a Tour Operator for example. Crafting a feasible travel plan is a whole different take. The last thing you want is to make mistakes there.\u003c/p\u003e\n\u003cp\u003eSo why would we let an AI do the talking to the customer and then replace it with a human? Isn\u0026rsquo;t it better to do it the other way around? Let our specialists use an AI to create a personal travel plan and send it quicker using AI? Bonding your customer with real human interaction is most preferable when doing sales.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eIf you dont know what you are talking about you are not able to spot mistakes in responses from AI. This will always be a major concern. First hand knowledge is still mandatory. That is why something like ChatGPT is a tool that needs to be used by skilled people about the subject to accelerate their work.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI dont believe in separate integrations within websites that take up space pretending to be a smart AI. The search function already present on websites will solve this issue. and people that are going to use an AI (LLM) dont want to interact with a different LLM every time. they will use their own PA to answer these questions if needed.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe first step companies need to solve is implementing a search engine in their website that understands intent. By understanding intent you can give people actually a useful answer based on the content on your website.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eYou need a profound integration with your LLM and your rich content. Especially in the Travel Industry. Using your website to become smarter using LLM is much more logical in my opinion then a popup \u0026ldquo;person\u0026rdquo; you can talk to.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eThe future of AI is undoubtedly exciting, but it\u0026rsquo;s a path we must navigate with both optimism and caution.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"my-take-on-ais-future\"\u003eMy Take on AI\u0026rsquo;s Future\u003c/h2\u003e\n\u003cp\u003eLooking ahead, I see AI as a field growing with possibilities but also filled with challenges. The journey from the early days of specific command-based systems to the more intuitive, context-aware models from today have been fascinating. However, there is still a long way to go in achieving the of AI that can really understand and interact with humans on a more profound level.\u003c/p\u003e\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eIn summary, while AI\u0026rsquo;s recent advancements may seem sudden and surprising to many, they are part of a long, ongoing process of technological evolution. My experiences at 333travel, especially with innovative platforms like Flow.ai, have shown me the potential and limitations of AI. As we move forward, I believe it\u0026rsquo;s crucial to balance our enthusiasm for AI with a realistic understanding of its current capabilities and limitations.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eYou are unique by keeping it close to yourself and not by using technology to do your core business for you. Always keep a personal touch!\u003c/p\u003e\n\u003c/blockquote\u003e\n","date_modified":"2024-02-16T08:44:46+01:00","date_published":"2024-01-23T22:58:00+01:00","id":"https://www.allsubjectsmatter.nl/why-ai-caught-you-by-surprise/","language":"en","summary":"Explore the AI (r)evolution. Discover how intent understanding transformed AI, from Flow.ai to ChatGPT, transforming the digital world in steps. Dive in for my own unique insights!","tags":["Strategy"],"title":"Why \"AI\" caught you by surprise","url":"https://www.allsubjectsmatter.nl/why-ai-caught-you-by-surprise/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eGrowing up, I always struggled with having wide feet, a challenge that only intensified over time. This culminated in 2023 when I underwent foot surgery to correct a bunion on my little toe. Before the surgery, wearing shoes often meant dealing with a painful pressure point caused by the bunion, a discomfort that increasingly invaded more of my footwear choices. The tipping point came when I could no longer enjoy inline skating for more than 45 minutes without experiencing significant pain.\u003c/p\u003e\n\u003cp\u003ePost-surgery, it became clear that my wide feet still posed a problem, particularly when it came to finding compatible inline skates. Despite my preference for hard-shell skates for their added support, I\u0026rsquo;ve found that they rarely accommodate wide feet comfortably. I\u0026rsquo;ve often been puzzled by claims that certain brands and models are suitable for wide feet, as none of the hard-shell skates I\u0026rsquo;ve tried have allowed me to skate without pain. Equally baffling is the industry\u0026rsquo;s oversight in not listing the width measurements of skate shells, considering that length alone doesn\u0026rsquo;t guarantee a good fit for those of us with wider feet.\u003c/p\u003e\n\u003cp\u003eIf manufacturers began to include width information, it would significantly ease the process of finding skates that fit well, thereby reducing the common dilemma of purchasing ill-fitting skates. This issue often leads to people abandoning the sport altogether. Offering boots that cater to those with slightly wider feet would allow for a more comfortable skating experience.\u003c/p\u003e\n\u003cp\u003eMoreover, providing clearer sizing information could reduce the frequency of returns to online shops, benefiting both retailers and customers. Retailers would face fewer losses from having to resell returned skates at lower prices, and customers would enjoy the satisfaction of owning skates that fit from the start. This change could create a win-win situation for everyone involved in the world of inline skating.\u003c/p\u003e\n\u003ch2 id=\"inline-skating-with-wide-feet\"\u003eInline skating with wide feet\u003c/h2\u003e\n\u003cp\u003eFollowing the surgery, I found that skating in my Roces Fifth Elements became significantly less painful in terms of the bunion. However, my feet still felt overly compressed within these skates, leading to general foot pain after skating for more than an hour. A brief respite by removing the skates for a few minutes offered some relief during longer sessions, but ultimately, I couldn\u0026rsquo;t wait to take them off at the end of the session.\u003c/p\u003e\n\u003cp\u003eThe Fifth Elements are often touted as the widest shells available, with many reviews suggesting a smaller size for a better fit, only for them to still feel too loose for many. However, labeling them as exceptionally wide seems misleading. While they may be wider than most, they fall short in delivering comfort, especially in the toe box area where individuals with wide feet, like myself, face the most discomfort. Despite the extra space in the heel, the toe box of the Fifth Element is still too narrow for comfort.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eOpinions are like assholes. Everybody has one.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI\u0026rsquo;ve attempted every possible modification to make my Roces Fifth Elements accommodate my wide feet better. From heat molding the shell to alleviate pressure points and widen them, to experimenting with different liners like the Roces RL2, and even trying Superfeet Carbon insoles for additional space. Despite these efforts and a significantly lighter wallet, I ended up with skates that still excessively compressed and caused too much pain in my feet.\u003c/p\u003e\n\u003ch2 id=\"technical-information\"\u003eTechnical information\u003c/h2\u003e\n\u003cp\u003eDelving into the technical specifics of my foot dimensions offers a clearer perspective on the challenges I\u0026rsquo;ve encountered. My feet measure approximately 11cm in width, with my right foot slightly wider at 11.5cm at its broadest point. Surprisingly, when consulting standard sizing charts, such as the one mentioned, my measurements place me within the \u0026lsquo;Extra Wide\u0026rsquo; category for a size 11, despite there being eight wider categories available. This revelation underscores the fact that, according to conventional sizing, my feet aren\u0026rsquo;t exceptionally wide.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.healthyfeetstore.com/pages/width-sizing-chart\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eShoe Width Measurement Chart | HealthyFeetStore.com\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eThe shoe width measurement chart from HealthyFeetStore.com can help ensure you find the best fit for your feet. Explore our foot width chart to learn more now.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/skating-with-wide-feet/bookmarks/01-icon_hu_df9bf35ed3f6de4a.webp\" alt=\"\" class=\"bookmark__icon\" width=\"32\" height=\"32\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eHealthyFeetStore.com\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/skating-with-wide-feet/bookmarks/01-thumb.svg\" sizes=\"250px\" alt=\"\" loading=\"lazy\" decoding=\"async\"\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cp\u003eThis discrepancy highlights a significant issue within the design of most inline skates. Despite claims of accommodating wider feet, the reality is that many skates do not offer the comfort promised for those of us with wider foot dimensions. In contrast, my experiences with standard footwear are markedly different.\u003c/p\u003e\n\u003cp\u003eBrands like Nike, Adidas, Fila, and Globe pose no problems for me; I can comfortably wear their shoes all day. This ease of fit with regular shoes accentuates the puzzling difficulty I face with hard-shell inline skates.\u003c/p\u003e\n\u003cp\u003eThe question then arises: Why is fitting into a hard shell skate so problematic compared to everyday footwear?\u003c/p\u003e\n\u003ch2 id=\"adapt-to-the-conclusion\"\u003eAdapt to the Conclusion\u003c/h2\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/skating-with-wide-feet/images/20231009_120808_hu_92a17cef684cb47.webp\" alt=\"\" width=\"560\" height=\"262\" data-zoom-src=\"https://www.allsubjectsmatter.nl/skating-with-wide-feet/images/20231009_120808_hu_774f6918a5c3e65.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003cp\u003eThe final solution to my skate fitting issues came in the form of custom skates crafted by \u003ca href=\"https://www.adaptbrand.com/\" data-external\u003eAdapt\u003c/a\u003e. Unfortunately, they\u0026rsquo;ve recently stopped producing the model I own, a custom Adapt Brutale size 43.5, tailored with a shell 3mm wider than their standard. These skates are a game-changer for me, fitting perfectly right out of the box. After heat molding, they became even more comfortable, as if custom-made for my feet.\u003c/p\u003e\n\u003cp\u003eThese skates changed my life completely. Finally skates i could fit like a glove. After heat molding them they got even better fitting!\u003c/p\u003e\n\u003cp\u003eReflecting on the journey, investing in these Adapt skates would have been more cost-effective than what I ended up spending on my Roces Fifth Elements. It\u0026rsquo;s a classic example of how hindsight is 20/20, but I don\u0026rsquo;t dwell on it too much. It\u0026rsquo;s all part of the learning curve that comes with pursuing a passion.\u003c/p\u003e\n\u003cp\u003eI want to extend a heartfelt shoutout to Adapt for their exceptional work. They continue to offer custom skates and are expected to unveil some new models soon. Their current pinnacle of custom skates is the \u003ca href=\"https://www.adaptbrand.com/masterpieces-collection\" data-external\u003eMasterpiece\u003c/a\u003e collection, which I am eagerly looking forward to acquiring in the future.\u003c/p\u003e\n\u003cp\u003eFor fellow skating enthusiasts, especially those with wider feet like mine, I cannot stress enough the value of investing in quality skates. It makes a world of difference in your skating experience. Support artisans and brands that exemplify true craftsmanship and passion for what they do. Your feet will thank you.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eA shoutout to Adapt to keep doing what they do best! Make skating an awesome experience!\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cfigure class=\"embed\"\u003e\u003ciframe src=\"https://www.youtube-nocookie.com/embed/_m-Qo-yiXKY\" title=\"Skates \u0026amp; Coffee - Our favourite models from the \u0026#39;Handmade\u0026#39; collection\" loading=\"lazy\" allow=\"accelerometer; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen referrerpolicy=\"strict-origin-when-cross-origin\"\u003e\u003c/iframe\u003e\u003c/figure\u003e\n\n\u003cblockquote class=\"pull-quote\"\u003e\u003c/blockquote\u003e\n","date_modified":"2024-02-15T13:54:34+01:00","date_published":"2024-01-11T21:39:00+01:00","id":"https://www.allsubjectsmatter.nl/skating-with-wide-feet/","language":"en","summary":"Wonder what skates would fit people with wider feet? Looking for good fitting skates? Read on and find my take on what you all find about this topic on the internet.","tags":["Inline skating"],"title":"Inline Skating for wider feet","url":"https://www.allsubjectsmatter.nl/skating-with-wide-feet/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eHere is the link to the original article with the waybackmachine: Due to covid and flow.ai beeing taken over by khoros this blog is no longer publicly available. This blog was written in q3 of 2019.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://web.archive.org/web/20201020201705/https://flow.ai/blog/how-333travel-is-revolutionizing-the-travel-industry\" data-external\u003ehttps://web.archive.org/web/20201020201705/https://flow.ai/blog/how-333travel-is-revolutionizing-the-travel-industry\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eSadly we have chosen to retire Joy due to the lack of results in early 2022 and Covid hitting us hard. The customer engagement was to low to keep it alive sadly.\u003c/p\u003e\n\u003chr\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eArtificial intelligence (AI) to keep in touch, in a personal way, outside opening hours. That is what \u003ca href=\"https://web.archive.org/web/20201020201705/https://www.333travel.nl/\" data-external\u003e333travel\u003c/a\u003e uses their chatbot they called Joy for.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe travel industry is an innovative market. The competition is prevalent and the urge for technological advantage is great but also difficult. If you do not operate technically independently in this market, there is a good chance that you will be absorbed by the crowd too much and can therefore no longer be unique in your proposition and your right to exist will be endangered.\u003c/p\u003e\n\u003cp\u003eWe are constantly looking to improve our position on the market and every day we translate IT into the business in the right way so that IT actually works for us. Keep experimenting and keep on trying new things!\u003c/p\u003e\n\u003ch2 id=\"our-challenge\"\u003eOur challenge\u003c/h2\u003e\n\u003cp\u003eWe are always looking for ways to better serve our customers and we want to do this in a personal way. This way we can really build a relationship with our customers and offer fully customized solutions.\u003c/p\u003e\n\u003cp\u003eWe are convinced that when you travel with us you will get the best possible experience of enjoyment because at 333travel we think along with you from A to Z and completely unburden you.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026hellip;With the help of Joy the customer has their answer faster and we can offer more quality because Joy indirectly ensures less work and more available information before a personal contact moment takes place\u0026hellip;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe use of artificial intelligence may therefore sound contradictory, but that is certainly not the case. We will of course start to use this AI to provide our customers with the existing information / FAQ on our website if they have any questions about it.\u003c/p\u003e\n\u003cp\u003eBut in addition to providing information, we will always try to direct to personal contact. It is important for us to physically come into contact with our potential customers, in a more personal, easy and accessible way.\u003c/p\u003e\n\u003ch2 id=\"our-goal\"\u003eOur goal\u003c/h2\u003e\n\u003cp\u003eThe automated collection of leads in a way that is as personal as possible is therefore our case for the implementation of this artificial intelligence. The rest is a added bonus. We strive for even more for personal contact through this AI. The focus is creating value and optimizing our customer journey outside of our regular working hours.\u003c/p\u003e\n\u003ch2 id=\"our-solution\"\u003eOur solution\u003c/h2\u003e\n\u003cp\u003eTogether with Flow.ai, we have started testing to see what personal ways we could use to get a customer to get in touch with us.\u003c/p\u003e\n\u003cp\u003eA correct use case of AI is very important here, but also the way in which you can train this AI. We wanted to create the most natural way of interaction, but we have to work with non sexy data such as “departure” and “arrival” dates to get an idea beforehand of when someone wants to leave for his holiday, this is a greater challenge than it seems.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eWe absolutely wanted to avoid that our AI does not immediately understand it when someone says “I want to visit Thailand on June 17 for 1 week”.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"a-natural-interaction-with-the-right-intent\"\u003eA natural interaction with the right intent\u003c/h2\u003e\n\u003cp\u003eWe have therefore trained our LMM that it understands intents with knowledge of what the departure date is and when you want to leave. It is therefore not an absolute fact what we want from you, but it is a value that falls within the meaning of “when do you want to leave”. There are so many different but also correct answers to this question and if you take our requirements into account, this bot must also understand everything below and more:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eIn the summer i want for the holidays…\u003c/li\u003e\n\u003cli\u003eDuring the winter i would like to stay …\u003c/li\u003e\n\u003cli\u003eIn autumn I want to go to…\u003c/li\u003e\n\u003cli\u003eI want to be in Thailand for Christmas…\u003c/li\u003e\n\u003cli\u003eWe want to go to … next year\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eIt immediately becomes unfriendly if after this question there is still a date field to be filled with when do you want to leave. Or even worse, the bot dont understands the question at all. a customer also often does not know when he wants to leave exactly. When he starts like this, is he obliged to fill in this information? You have one try with this type of customer.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eCustomers are fed up enough already with non working chatbots from large companies trying to reduce FTE and see how far they can go to annoy their customers.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eWe instantly record the key information of the expected departure period automatically. After all, this will be deepened further in the contact moment, for now our specialists will know what the intent is and can act accordingly.\u003c/p\u003e\n\u003cp\u003eIt is also possible within our implementation to capture all the variables we need with only one sentence from a customer. We only need the contact details after this and we give a call on the requested time and date.\u003c/p\u003e\n\u003ch3 id=\"collecting-data\"\u003eCollecting data\u003c/h3\u003e\n\u003cblockquote\u003e\n\u003cp\u003e“Good afternoon, I am looking for a holiday starting in Indonesia and ending in Bali for 2 adults and 2 children with a hotel on the beach leaving July 21 and coming home the 3th of August .”\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe above sentence is a practical example of what Joy understands. We record these values and we immediately know that there is enough data. We only ask for the customer’s telephone number/e-mail address and call him at a preferred time specified by himself.\u003c/p\u003e\n\u003cp\u003ewe call you at the preferred moment, we consciously chose not to let the customer choose something through a date picker to keep the natural flow. For the customer it is easier to say: “next Thursday at 10 am”. From this, we understand what the intent is and call that particular customer at that exact time. There are also plenty of customers who say they want to be called right now. Regardless of whether they can be called at that moment or not, they drop out as soon as they get a date picker presented. Especially when the present day is missing.\u003c/p\u003e\n\u003cp\u003eBecause the customer visits our website, we also know from which page we have received his request for personal contact. This way we have even more information for our specialists to get an idea of what the intent is.\u003c/p\u003e\n\u003cp\u003eThis data gets send automatically as a lead/proposal in our backoffice called BOSS. And every follow up on this lead is recorded there.\u003c/p\u003e\n\u003ch2 id=\"result-joy-sees-the-light\"\u003eResult: Joy sees the light.\u003c/h2\u003e\n\u003cp\u003eAfter a large number of internal tests, various tweak adjustments and using various methods to achieve our goal, Joy was born. We have also given her a personal face because of our intention to keep the conversations personal. Of course, we do indicate that Joy is a “Virtual Travel Assistant”. Our real colleagues are still the travel specialists.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u0026hellip;What is particularly noticeable from the results is that the interactions and requests that come in via Joy are usually from people between 20 - 35 years old\u0026hellip;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"important-lessons\"\u003eImportant lessons\u003c/h2\u003e\n\u003cp\u003eThe most important lesson in this might be a bit of an open door, but make sure your goal is set on paper, and start with the simplest implementation of this goal and slowly develop it into a more complex working final product.\u003c/p\u003e\n\u003cp\u003eWe first started to look at ways in which we can collect our required data with \u003ca href=\"https://web.archive.org/web/20201020201705/https://flow.ai/\" data-external\u003eFlow.ai\u003c/a\u003e and really understand how this works. Then we tested a lot of different scenario\u0026rsquo;s and slowly worked towards one complete flow that combined the best of multiple flows together. We immediately gained a lot of experience with the various options that flow provided this way.\u003c/p\u003e\n\u003cp\u003eWe are well supported in this by Flow.ai, but due to our own in-house knowledge, we didn’t require the support all the time and could really dive deep on the subject.\u003c/p\u003e\n\u003cp\u003eThe flow.ai system is well structured, very logical and understandable. This shouldn’t block you from experimenting.\u003c/p\u003e\n\u003ch2 id=\"the-future\"\u003eThe future\u003c/h2\u003e\n\u003cp\u003eThe first Quarter is our busiest period. Because we implemented Joy relatively shortly before this period, we have not yet activated the option to Live Chat. This is something that is going to be activated after this quarter.\u003c/p\u003e\n\u003cp\u003eIn addition, we want to go even further in the near future with Joy and make her THE assistant of our customer portal that will help you on your way after you made a booking. If Joy does not know the answer, you will immediately get in touch with the specialist you booked the trip at. If you have any questions during your trip? Ask Joy. Do you have problems during your trip? Joy ensures that you get the answer as soon as possible and that you get into direct contact with us when needed.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eWe see Joy as our “Virtual Travel Assistant” where the very specific knowledge of our destinations and our knowledge of customization remains with our travel specialists because this is the strength of our personal contact in which we personally make the difference, within our philosophy.\u003c/p\u003e\n\u003c/blockquote\u003e\n","date_modified":"2024-01-24T14:00:17+01:00","date_published":"2024-01-02T13:48:00+01:00","id":"https://www.allsubjectsmatter.nl/how-333travel-is-revolutionizing-the-travel-industry/","language":"en","summary":"Discover how 333travel's AI 'Joy' redefines customer interaction back in 2019, blending AI's intent understanding with personal touch. A game-changing journey in AI-led travel solutions!","tags":["Strategy","Building"],"title":"2019 - Blast from the past: How 333travel is revolutionizing the Travel industry","url":"https://www.allsubjectsmatter.nl/how-333travel-is-revolutionizing-the-travel-industry/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"some-background-to-understand-our-issue-better\"\u003eSome background to understand our issue better\u003c/h2\u003e\n\u003cp\u003eWe are working on a new website with a great emphasis on images. We already have \u003ca href=\"http://www.333travel.nl/\" data-external\u003ewww.333travel.nl\u003c/a\u003e which of course being a travel product already heavily relies on images. Our new website has even higher res images loaded because there is a great emphasis on the luxury feel. This site is aimed towards the luxury traveller \u003ca href=\"http://www.travelplatinum.nl/\" data-external\u003ewww.travelplatinum.nl\u003c/a\u003e. The project that served our images static.333travel.nl is already a few years running with no reported issues for the 333travel website.\u003c/p\u003e\n\u003ch2 id=\"the-issue\"\u003eThe issue\u003c/h2\u003e\n\u003cp\u003eThe issue at hand got reported from our home workers. They noted that when working from home a lot of the images displayed were sometimes broken on TravelPlatinum (and a refresh mostly fixed them). They also replied that it was all working fine when working from within the office. So that made us start a quest in finding the culprit.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/sudden-issues-loading-images-from-our-static-project/images/data-src-image-8526b527-c281-496a-9251-1028b11d3560_hu_1bb3200cf0522d93.webp\" alt=\"\" width=\"560\" height=\"382\" data-zoom-src=\"https://www.allsubjectsmatter.nl/sudden-issues-loading-images-from-our-static-project/images/data-src-image-8526b527-c281-496a-9251-1028b11d3560_hu_220750cfd07c6574.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003cp\u003eWe noticed that in the browsers we use (Edge and Chrome) there were reports from the failed images in the console which reported. ERR_HTTP2_PROTOCOL_ERROR. With that message in the pocket we started our search for the issue searching through the logs on the server. We are running Nginx as our webserver.\u003c/p\u003e\n\u003cp\u003eWe quickly found a trail that appeared for every image that we thought didn’t load correctly. Since we weren’t able to simulate the issue ourselves yet.\u003c/p\u003e\n\u003cp\u003e0.0.0.0 - - [06/Nov/2023:15:45:02 +0100] \u0026ldquo;GET /web-images/10/2023/653775730fc66/Jazan\u0026mdash;Al-qahar-mountains-1200.webp HTTP/2.0\u0026rdquo; 206 183064 \u0026ldquo;\u003ca href=\"https://www.travelplatinum.nl/\" data-external\u003ehttps://www.travelplatinum.nl/\u003c/a\u003e\u0026rdquo; \u0026ldquo;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0\u0026rdquo;\u003c/p\u003e\n\u003cp\u003ePay attention to the 206 call here. This means partial content loaded.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/206\" data-external\u003ehttps://developer.mozilla.org/en-US/docs/Web/HTTP/Status/206\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eThis was odd since our webserver didn’t support partial loading of images. We never send headers that replied with something like a range which are required to be able to do partial content.\u003c/p\u003e\n\u003ch2 id=\"our-attempt-to-fix-the-issue\"\u003eOur attempt to fix the issue\u003c/h2\u003e\n\u003cp\u003eSo that was the first issue we dove in. I found out that Nginx by default sends a header that replies Content-Type: bytes. I don’t know why I would want our image server to respond to a 206 request with an ok message. So we disabled this with the following setting in our server block.\u003c/p\u003e\n\u003cp\u003eserver {\u003cbr\u003e\nserver_name static.333travel.nl;\u003cbr\u003e\nmax_ranges 0;\u003cbr\u003e\n}\u003c/p\u003e\n\u003cp\u003eWe replied back to the persons experiencing this issue to report to us if it happened again. Which sadly was quickly the case. The error message did change in our logging. Edge and chrome kept spitting out the same useless ERR_HTTP2_PROTOCOL_ERROR in the console tab. In the timeline we also saw these images stopping to load at a certain point.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/sudden-issues-loading-images-from-our-static-project/images/data-src-image-1cdc5aec-ad55-4bfd-8a7c-2593b57282be_hu_597e0975f3b81981.webp\" alt=\"\" width=\"560\" height=\"247\" data-zoom-src=\"https://www.allsubjectsmatter.nl/sudden-issues-loading-images-from-our-static-project/images/data-src-image-1cdc5aec-ad55-4bfd-8a7c-2593b57282be_hu_ccaba7f5f6ea432e.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003cp\u003e0.0.0.0 - - [08/Nov/2023:14:22:27 +0100] \u0026ldquo;GET /web-images/06/2020/5ed7ad48d3d79/mexico-yucatan-chichen-itza-ik-kil-cenote.jpeg HTTP/2.0\u0026rdquo; 200 0 \u0026ldquo;\u003ca href=\"https://vue-boss.333office.nl/\" data-external\u003ehttps://vue-boss.333office.nl/\u003c/a\u003e\u0026rdquo; \u0026ldquo;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eWe now get a header with a length of 0 bytes and a response with http 200 (ok). This is getting real strange here. The header when inspected in the browser returns the right amount of bytes in the response header. So why does the log of Nginx state it replies with a size of 0. This is where things got weird and we didn’t really know what to do next. We first had to be able to reproduce the issue ourself so we could really start to troubleshoot.\u003c/p\u003e\n\u003cp\u003eI tried to simulate a fault with ab (apache benchmark). I started conservative with 500 requests and 50 concurrent requests. The issue never triggered for me. i then moved in small steps upwards to end on 10.000 requests with 700 concurrent requests without ever triggering the reported issue. The server load also appears to handle everything fine. It was an image of around 500kb i requested there.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/sudden-issues-loading-images-from-our-static-project/images/image_hu_e4a7c89173cb9060.webp\" alt=\"\" width=\"560\" height=\"339\" data-zoom-src=\"https://www.allsubjectsmatter.nl/sudden-issues-loading-images-from-our-static-project/images/image_hu_52a1f89bab47586c.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"the-facts\"\u003eThe Facts\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003eSetting a slow profile in the network tab when inspecting didn’t simulate the issue for us.\u003c/li\u003e\n\u003cli\u003eThe error was still only reported working from home from 2 users.\u003c/li\u003e\n\u003cli\u003eWhen in the office they haven’t had a single issue.\u003c/li\u003e\n\u003cli\u003eLoading the site 333travel.nl didn’t report the issue at all. Every image worked there also for the 2 persons working from home.\u003c/li\u003e\n\u003cli\u003eUsing a 5g connection testing from all various places we could think off didn’t simulate the issue once for me.\u003c/li\u003e\n\u003cli\u003eUsing ab (apache benchmark) doing 10.000 requests with 700 concurrent requests never triggered the issue at hand.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eThere was still one difference and that was the domain serving the images. 333travel got replied with a Cross origin header of same_site which of course is true since the domain is the same. So I added a header to the static.\u003c/p\u003e\n\u003cp\u003eserver {\u003cbr\u003e\nserver_name static.333travel.nl;\u003cbr\u003e\nmax_ranges 0;\u003cbr\u003e\nadd_header \u0026lsquo;Access-Control-Allow-Origin\u0026rsquo; \u0026lsquo;*\u0026rsquo;;\u003cbr\u003e\n}\u003c/p\u003e\n\u003cp\u003eThis way every source wanting to use images from our static.333travel.nl gets the same reply. This didn’t solve anything about the issue reported.\u003c/p\u003e\n\u003cp\u003eWe still experienced issues to simulate the issue ourself. We were pretty sure that this only got triggered on a somehow bad connection visiting TravelPlatinum. Visiting 333travel which served the images from the same static server never showed the issue we are experiencing on TravelPlatinum.\u003c/p\u003e\n\u003ch2 id=\"simulating-a-bad-wifi-connection\"\u003eSimulating a bad WiFi connection\u003c/h2\u003e\n\u003cp\u003eWe badly wanted to simulate a bad WiFi connection. I moved myself to a spot where there was bad WiFi outside of our office and I started to finally see the issue for myself. Finally something to start our testing on. While it was still odd we needed a somehow bad/slow WiFi to simulate this. You would say setting your profile to slow 3g in the browsers developer tools would behave the same.\u003c/p\u003e\n\u003cp\u003eNow we finally were able to start testing from our development environment. We started to suspect the project itself doing something weird with maybe something javascript related closing the connection or something. So we started to create a stripped down website from TravelPlatinum to test without javascript.\u003c/p\u003e\n\u003cp\u003eThe no image loading issue was still there..\u003c/p\u003e\n\u003cp\u003eWe decided to take in the exact same html from 333travel.nl to test with that to create a same environment. Since 333travel still was going strong every refresh.\u003c/p\u003e\n\u003cp\u003eThe no image loading issue was still there..\u003c/p\u003e\n\u003cp\u003eWe badly wanted to be able simulate this even better since our bad WiFi spot wasn’t at a desk or something, we were sitting on a bunch of stairs debugging the issue from there. I decided to create a 2\u003csup\u003end\u003c/sup\u003e WiFi hotspot in our office with a throttled connection of 3mbit and see where that takes us. This was it! We could now simulate every refresh a lot of broken images on TravelPlatinum! We are getting somewhere at least. Bear in mind that the 333travel website still was loading every image with no issues.\u003c/p\u003e\n\u003cp\u003eWe moved away from the nginx logs at this moment in time since we uncovered that going in that direction nothing was going to be solved at this moment. Nginx still replied an ok on a header with 0 bytes. If that’s ok for nginx with the right headers in place, that message is not going to help us solve the issue at this moment.\u003c/p\u003e\n\u003ch2 id=\"testing-with-firefox\"\u003eTesting with Firefox\u003c/h2\u003e\n\u003cp\u003eThe moment I created this WiFi hotspot we were just about ready to test images with just loading the html image tag and nothing more loaded in the project. This still broke our images. I then suggested to fire up Firefox just to see if that behaved differently since I suspected it to be a browser related thing on how it responds on what is happening. Or at least it could output maybe a different error. Something we could focus our search for the solution on.\u003c/p\u003e\n\u003cp\u003eWe saw that Firefox was indeed loading the images line by line on the 3mbit connection. But it suddenly stopped loading leaving broken images in place which were about 50% loaded or not at all. This was finally something of a change to see! Still Edge/Chrome displayed no image at all and gave me a useless error message. I now finally started to realize that the static was actually sending the images and the browser received them but the connection got closed to soon somehow.\u003c/p\u003e\n\u003ch2 id=\"the-solution\"\u003eThe solution\u003c/h2\u003e\n\u003cp\u003eAnd that was when I finally found the issue at hand. A header i already set ages ago on a global level in Nginx in a performance tuning session.\u003c/p\u003e\n\u003cp\u003esend_timeout 2;\u003c/p\u003e\n\u003cp\u003eWhat send timeout in my opinion does is keeping the connection open for X seconds after not receiving data. The default is 60 seconds and I adjusted it to 2 seconds so we would not have open connections for a whole 60 seconds doing nothing.\u003c/p\u003e\n\u003cp\u003eWhy would that be the issue since I clearly saw Firefox visually loading the image line by line? There is constant data being send then right? Why would this setting then be related to closing the connection.\u003c/p\u003e\n\u003ctable\u003e\n\t\u003cthead\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003cth\u003eSyntax:\u003c/th\u003e\n\t\t\t\t\t\u003cth\u003e\u003cstrong\u003esend_timeout\u003c/strong\u003e \u003cem\u003etime\u003c/em\u003e;\u003c/th\u003e\n\t\t\t\u003c/tr\u003e\n\t\u003c/thead\u003e\n\t\u003ctbody\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003ctd\u003eDefault:\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003esend_timeout 60s;\u003c/td\u003e\n\t\t\t\u003c/tr\u003e\n\t\t\t\u003ctr\u003e\n\t\t\t\t\t\u003ctd\u003eContext:\u003c/td\u003e\n\t\t\t\t\t\u003ctd\u003ehttp, server, location\u003c/td\u003e\n\t\t\t\u003c/tr\u003e\n\t\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003cem\u003eSets a timeout for transmitting a response to the client. The timeout is set only between two successive write operations, not for the transmission of the whole response. If the client does not receive anything within this time, the connection is closed.\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eWell it appears this does get triggered on slower connections. This is getting interpreted as a stalled request by Nginx. And it closes the whole connection to in our case the static.333travel.nl. So the partially loaded image and every image after that not being loaded wont get loaded again since the connection to static is closed.\u003c/p\u003e\n\u003cp\u003eWhy simulating a slow connection in the developer tools wont trigger this issue is still something I don’t know for sure. I have some theories, a bad WiFi also introduces more latency and lost packets getting resend. But I don’t know the real reason. I dont know how these browsers simulate a slow connection.\u003c/p\u003e\n\u003cp\u003eSetting this send_timeout to 8 seconds as below solved our issue completely.\u003c/p\u003e\n\u003cp\u003eserver {\u003cbr\u003e\nserver_name static.333travel.nl;\u003cbr\u003e\nmax_ranges 0;\u003cbr\u003e\nadd_header \u0026lsquo;Access-Control-Allow-Origin\u0026rsquo; \u0026lsquo;*\u0026rsquo;;\u003cbr\u003e\nsend_timeout 8;\u003cbr\u003e\n}\u003c/p\u003e\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eWe have learned a lot, and we solved the issue. The odd thing remains that all this time it only triggered on our TravelPlatinum project. We were obviously balancing on the limit with the send_timeout setting since the image load is greater on TravelPlatinum.\u003c/p\u003e\n\u003cp\u003eAnother important thing to note, setting a slow connection in your developer tools is not a real simulation from a slow connection. You can simulate all you want but you need real life cases to really measure if everything is working fine.\u003c/p\u003e\n","date_modified":"2024-02-16T08:45:11+01:00","date_published":"2023-11-12T12:50:00+01:00","id":"https://www.allsubjectsmatter.nl/sudden-issues-loading-images-from-our-static-project/","language":"en","summary":"We noticed that in the browsers we use (Edge and Chrome) there were reports from the failed images in the console which reported. ERR_HTTP2_PROTOCOL_ERROR","tags":["Under the hood"],"title":"Issues displaying images from our Static server","url":"https://www.allsubjectsmatter.nl/sudden-issues-loading-images-from-our-static-project/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"notepad-regexp\"\u003eNotepad++ regexp\u003c/h2\u003e\n\u003cp\u003eFor example checking your php-fpm access/slow log file. Use mark in notepad++ to mark all lines that are slower then x/y.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-text\"\u003e\u003ccode class=\"language-text\"\u003e10000ms en trager \n\n^.*\\s(\\d{5,}\\.\\d{3}ms).*\n\n5000ms en trager \n\n^.*\\s(5\\d{3}\\.\\d{2,}ms|\\d{5,}\\.\\d{2,}ms).*\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch2 id=\"dns-tools-to-automatically-check-the-transfer-of-your-dns\"\u003eDNS tools to automatically check the transfer of your DNS\u003c/h2\u003e\n\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003esudo apt update\nsudo apt install dnsutil\n\nnano check_nameservers.sh\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003e#!/bin/bash\n\nDOMAIN=\u0026amp;#34;jouwdomein.com\u0026amp;#34;\nEXPECTED_NS1=\u0026amp;#34;ns1.cloudflare.com.\u0026amp;#34;\nEXPECTED_NS2=\u0026amp;#34;ns2.cloudflare.com.\u0026amp;#34;\n\n# Get the current nameservers\nCURRENT_NS=$(dig NS $DOMAIN \u0026#43;short)\n\n# Check if the current nameservers match the expected ones\nif [[ $CURRENT_NS == *\u0026amp;#34;$EXPECTED_NS1\u0026amp;#34;* \u0026amp;amp;\u0026amp;amp; $CURRENT_NS == *\u0026amp;#34;$EXPECTED_NS2\u0026amp;#34;* ]]; then\n    echo \u0026amp;#34;Nameservers are correctly set to Cloudflare.\u0026amp;#34;\nelse\n    echo \u0026amp;#34;Nameservers have not yet switched to Cloudflare.\u0026amp;#34;\nfi\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cpre class=\"line-numbers language-bash\"\u003e\u003ccode class=\"language-bash\"\u003echmod \u0026#43;x check_nameservers.sh\n\nwatch -n 1 ./check_nameservers.sh\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e","date_modified":"2024-09-11T09:24:35+02:00","date_published":"2023-09-11T09:21:00+02:00","id":"https://www.allsubjectsmatter.nl/useful-commands/","language":"en","summary":"Useful commands and scripts that simplify technical tasks, like managing logs using Notepad++ with regex or automating DNS checks for domains.","tags":["Under the hood"],"title":"Useful commands","url":"https://www.allsubjectsmatter.nl/useful-commands/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eI first learned about Scrum way before the 2020 manifesto update was in place. So when we went to a Scrum master training after this 2020 release we had a nice little discussion about the change that has had the most impact with me since this update to the Scrum guide.\u003c/p\u003e\n\u003cp\u003eThe shift from creating one inspectable working artifact at the end of a sprint. Into releasing much more frequently within the sprint itself. This is a real good thing since we live in a CI/CD world and holding something back when its Done is not necessary.\u003c/p\u003e\n\u003cp\u003eThe problem i have with this is that it shifted the focus from my knowledge about what a Sprint Review was. And this got me thinking.\u003c/p\u003e\n\u003ch2 id=\"some-more-background\"\u003eSome more background\u003c/h2\u003e\n\u003cp\u003eFor our team, achieving the \u0026ldquo;Done\u0026rdquo; status means a feature is primed for deployment. This doesn\u0026rsquo;t imply hurried releases; substantial changes are deliberated upon, ensuring they align with user requirements and broader objectives. Each update is meticulously vetted, with our Product Owner playing a pivotal role, aligning the release with business imperatives.\u003c/p\u003e\n\u003cp\u003eIf you have a decent CI/CD implementation your team can forget that this update is released the moment it is deployed in our staging area. When we decide to pull the update trigger and go live is up to the business. The team will need to know when something is fully released since we also keep an eye on the potential issues. But the Developers should not have a stake in the actual release. I understand this needs some more explanation. We save that post for the future.\u003c/p\u003e\n\u003cp\u003eWe do keep the DONE unreleased and DONE released separated in our DONE lane.\u003c/p\u003e\n\u003ch2 id=\"our-mini-sprint-reviews\"\u003eOur mini Sprint Reviews\u003c/h2\u003e\n\u003cp\u003eTo draw a finer point, our feature releases often resemble a series of \u0026ldquo;mini Sprint Reviews\u0026rdquo;\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThere\u0026rsquo;s a comprehensive discussion about changes and potential impacts.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eDeployment timings are strategized.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eDetailed changelogs are curated.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eStakeholders are kept in the loop, their feedback actively sought post-release.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThis thorough approach often means that by the time our Sprint Review rolls around, the major talking points have been addressed.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThis takes away some topics from the sprint review. Especially when you have a decent CI/CD in place and you work by frequent releases.\u003c/p\u003e\n\u003cp\u003eWhen you discuss and go live with updates before the end of the sprint and you got the feedback on the released features. Why bother people again to provide feedback of discuss the release again? Scrum is a framework that facilitates talk. But we do need value in this meetings. And to us the sprint review on this topic has become quite useless.\u003c/p\u003e\n\u003ch2 id=\"what-do-we-discuss\"\u003eWhat do we discuss?\u003c/h2\u003e\n\u003cp\u003eOne might wonder, with such an exhaustive pre-release evaluation, what remains for the Sprint Review? We\u0026rsquo;re refining our approach, but here\u0026rsquo;s our current structure\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAssessing our adherence to the sprint goal.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAnalyzing tasks that remain untouched in the sprint.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eDelving into ongoing development items.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eAdjusting the backlog to reflect any changes from the last sprint, ensuring upcoming sprints have clear priorities.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eProactively identifying new tasks or adjustments based on our recent learnings.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eEngaging stakeholders for detailed feedback where necessary.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eWorking like this offers some distinct advantages for us. Stakeholders are in all departments of our company. A lot of them work hybrid or even 3/4 days in a hybrid combination. By doing frequent releases and planning accordingly we discuss the release when they are in the company which just works best for us. The ability to discuss things face to face is invaluable. It just works better and creates more understanding between each other.\u003c/p\u003e\n\u003cp\u003eWhile our reviews traditionally fall on Fridays. This is not an issue since we can just notify the stakeholder any day we are done. Discuss the update. And go live. We dont have to make a whole lot of fuzz about these things. We have short feedback loops! Planning dreadful meetings is for slow moving large companies where everybody thinks they are important and keeps stalling things in bureaucracy.\u003c/p\u003e\n\u003cp\u003eMost of our Sprint Reviews dont discuss anything related to what we deployed. We dont inspect the increment. The increment is already inspected and we move on quickly to the other topics as discussed above.\u003c/p\u003e\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eA few essential components drive this approach of ours.\u003c/p\u003e\n\u003cp\u003eA Product Owner with genuine decision-making authority, ensuring alignment with broader business goals.\u003c/p\u003e\n\u003cp\u003eYou need a Scrum team which works tightly together in completing full stories or features. Getting them really DONE. You need FLOW in your team to be able to pull this off.\u003c/p\u003e\n\u003cp\u003eYou need a decent CI/CD framework in which you can deploy automated and frequently. (We are working on that ourselves. We are automated but not enough just yet. We can take out a lot more manual labor.)\u003c/p\u003e\n\u003cp\u003eAs Scrum continues to evolve, so do our practices, always keeping value and efficiency at the forefront.\u003c/p\u003e\n","date_modified":"2024-02-16T08:45:17+01:00","date_published":"2023-08-29T08:12:00+02:00","id":"https://www.allsubjectsmatter.nl/sprint-reviews-in-scrum/","language":"en","summary":"I first learned about Scrum way before the 2020 manifesto update was in place. So when we went to a Scrum master training after this 2020 release we had a nice little discussion about the change that has had the most impact with me since this update to the Scrum guide.","tags":["Teams"],"title":"Sprint Reviews in high performance Scrum","url":"https://www.allsubjectsmatter.nl/sprint-reviews-in-scrum/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eDiscover my personal rollerblading routes in Alphen aan den Rijn via \u003ca href=\"https://www.google.nl/maps/d/u/1/edit?mid=1fG1tn4gzgQ-jEQOcD-t-NTvVzb4BUoI\u0026amp;usp=sharing\" data-external\u003eGoogle My Maps\u003c/a\u003e. These carefully chosen paths have an excellent surface for at least 90% of the route. While I do my best to find the ideal routes, you may occasionally run into some rougher asphalt, cobbles, and cattle grids.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eWhat you can expect from my rollerblading routes:\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eDistances ranging between 5 and 25 km.\u003c/li\u003e\n\u003cli\u003eNew routes added regularly to complement the existing ones.\u003c/li\u003e\n\u003cli\u003eClear start and end points with parking for both bikes and cars.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"de-routes\"\u003eThe routes\u003c/h2\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.google.nl/maps/d/u/1/edit?mid=1fG1tn4gzgQ-jEQOcD-t-NTvVzb4BUoI\u0026amp;usp=sharing\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eInline Skateroutes Alphen aan den Rijn - Google My Maps\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eRoutes die ik geskate heb en prima te doen zijn qua asfalt. Let wel op dat specifieke routes veeroosters kunnen hebben!\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/inline-skating-routes-rond-alphen-aan-den-rijn/bookmarks/01-icon.ico\" sizes=\"25px\" alt=\"\" class=\"bookmark__icon\" loading=\"lazy\" decoding=\"async\"\u003e\u003cspan\u003eGoogle My Maps\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/inline-skating-routes-rond-alphen-aan-den-rijn/bookmarks/01-thumb_hu_73c765fb71066390.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cp\u003eVia the link above you can find the routes I\u0026rsquo;ve rollerbladed. Use my routes as inspiration for your next rollerblading adventure in and around Alphen aan den Rijn!\u003c/p\u003e\n\u003ch3 id=\"route-rietveldse-pad\"\u003eRoute: Rietveldse pad.\u003c/h3\u003e\n\u003cp\u003eYou can easily park your car or bike at the starting point, or rollerblade there yourself. We start from the parking lot as shown and end there as well. Perfect for a relaxed ride.\u003c/p\u003e\n\u003ch3 id=\"route-rietveldse-pad-optie-2-let-op-veeroosters\"\u003eRoute: Rietveldse pad option 2 (Watch out! Cattle grids).\u003c/h3\u003e\n\u003cp\u003eThere are cattle grids on this route. You\u0026rsquo;ll find at least 2 of them. They\u0026rsquo;re located between marker C and D. Parking is easy here too, and you finish where you start.\u003c/p\u003e\n\u003ch3 id=\"route-rietveldse-pad-naar-hazerswoude\"\u003eRoute: Rietveldse pad to Hazerswoude.\u003c/h3\u003e\n\u003cp\u003eHere too you can easily park your car or bike at the starting point, or rollerblade there yourself. We start from the well-known parking lot and end there as well.\u003c/p\u003e\n\u003ch3 id=\"route-rietveldse-pad-naar-hazerswoude-alternatief\"\u003eRoute: Rietveldse pad to Hazerswoude alternative.\u003c/h3\u003e\n\u003cp\u003eEasy to park your car or bike at the starting point. We start from the parking lot and end there as well.\u003c/p\u003e\n\u003ch3 id=\"route-korte-ridderbuurt-heen-en-weer\"\u003eRoute: Korte Ridderbuurt out and back.\u003c/h3\u003e\n\u003cp\u003eGreat route to do some carving and improve your skills. You can easily go back and forth multiple times :). Little car traffic and beautiful views over the farmland.\u003c/p\u003e\n\u003ch3 id=\"route-ter-aar-korteraar-en-langeraar-let-op-veeroosters\"\u003eRoute: Ter Aar, Korteraar and Langeraar (Watch out! Cattle grids).\u003c/h3\u003e\n\u003cp\u003eYou\u0026rsquo;ll find cattle grids in multiple places here! Between F and G you\u0026rsquo;ll find at least 2 cattle grids. And between H and I on the Windepad you\u0026rsquo;ll find at least 6 cattle grids. The longest route so far, and otherwise great to rollerblade.\u003c/p\u003e\n\u003ch3 id=\"route-richting-boskoop-via-rietveldse-pad-terug\"\u003eRoute: Towards Boskoop via Rietveldse pad back.\u003c/h3\u003e\n\u003cp\u003eThis route is still partly under exploration, but I\u0026rsquo;ve cycled it regularly and already rollerbladed parts of it. This route looks very doable and is another combination with the Rietveldse pad. The stretch towards Boskoop and along the station is all fine. The Raoul Wallenbergplein is also easy to reach by car or bike. Parking is easy in parking garage Castellum.\u003c/p\u003e\n\u003cp\u003eKeep in mind that every inline skater has different preferences. What I find ideal may not be for you at all.\u003c/p\u003e\n","date_modified":"2023-10-11T10:35:25+02:00","date_published":"2023-08-14T08:00:00+02:00","id":"https://www.allsubjectsmatter.nl/inline-skating-routes-rond-alphen-aan-den-rijn/","language":"en","summary":"Discover my personal skate routes in Alphen aan den Rijn via Google My Maps. These carefully chosen paths have an excellent surface for at least 90% of the way. While I do my best to find the ideal routes, you may occasionally run into some rougher asphalt, cobblestones, and cattle grids.","tags":["Inline skating"],"title":"Inline skate routes Alphen aan den Rijn","url":"https://www.allsubjectsmatter.nl/inline-skating-routes-rond-alphen-aan-den-rijn/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eFirst of all I want to say this.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eOpinions are like assholes. Everybody has one\u0026hellip;\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI wont go any further but there are a lot of spinoffs for this quote. Do read on with a little skepticism.\u003c/p\u003e\n\u003ch2 id=\"the-word-sprint\"\u003eThe word \u0026ldquo;Sprint\u0026rdquo;\u003c/h2\u003e\n\u003cp\u003eI keep having some difficulties using the word Sprint for every 2 week cycle of our product every time i think of what the word means to me. These sprints never stop! The moment a Sprint ends a new sprint starts\u0026hellip;\u003c/p\u003e\n\u003cp\u003eIn real life you at least can catch your breath and do some sleeping before travelling towards a new Sprint. And no a retro isn\u0026rsquo;t good enough. \u003cem\u003e(SCRUM police)\u003c/em\u003e A new sprint starts the moment a sprint ends! The retro is also done within a sprint. \u003cem\u003e(/SCRUM police)\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eI am a native Dutch speaker so my interpretation of the word sprint might differ. But to me a sprint instills going as fast as you can to finish first. Of course you obey the rules that wont disqualify you from winning the sprint. But the word rush comes into mind every time I think of a Sprint. Finish it! no matter what. This meaning gives a feeling of haste which isn’t gonna deliver cleaner code or less technical debt.\u003c/p\u003e\n\u003cp\u003eAfter finishing a sprint in real life I know i\u0026rsquo;m not physically capable of starting another sprint immediatly. So if you want to get in a flow as a team you have to actually stop rushing and start the discussion by talking to eachother. Take the time to make clear everyone is on the same page and we all see the same goal.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eAfter all, SCRUM is a talk facilitating framework isnt it?\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe word \u0026ldquo;Sprint\u0026rdquo; also means individuality to me. Which is unwanted considering your are a SCRUM team trying to get in a groove or flow together.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/achieving-flow-with-scrum/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eAchieving FLOW with Scrum.\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eIn this post I share my current journey of implementing Scrum and Kanban principles with my team, this is mainly about emphasizing the importance of real collaborative work and instilling a product vision which helps us to achieve FLOW which translates to high value Scrum for us.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/scrum-sprint-misinterpretations-within-scrum/bookmarks/01-icon_hu_9b466dd874965fcb.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco Loup.\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/scrum-sprint-misinterpretations-within-scrum/bookmarks/01-thumb_hu_c670a19d35689e3b.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cp\u003eJust goofing around, but calling it an relay race (estafette) can be a better name! This also instills handovers and teamwork to be done with each other. The result is the same for the whole team if you compare it to a real life relay race. But still the aspect haste or rush is present here.\u003c/p\u003e\n\u003cp\u003eYou of course don’t have to change this name but discussing this topic with your team is a thing you can do! Discuss this matter together with your, DOD, Acceptance Criteria and Sprint goal. At least make sure everyone understands that you don’t want the rush aspect that comes with the term sprint. Better do it the best you can obeying all defined rules. Instead of cutting corners in some grey area where nobody sees you do it!\u003c/p\u003e\n\u003ch2 id=\"change-that-name\"\u003eChange that name!\u003c/h2\u003e\n\u003cp\u003eAnd if we need another name. We can maybe use Development Cycles? This way we can call it Cycles like \u003ca href=\"https://linear.app/\" data-external\u003eLinear\u003c/a\u003e does. Or call it Flows. For our 2 week Flow I have this Flow goal in mind.\u003c/p\u003e\n\u003cp\u003eFor a name change to work we have to all agree on using this name. Tools like Jira already define a lot of these names that aren’t changeable. We as the development team are in a lucky position where Jira is only used for our backlog and administration purposes. But our physical Scrum board is our single source of truth of what we are trying to accomplish.\u003c/p\u003e\n\u003cp\u003eOur sprints only get registered on a higher level in Jira. All daily tasks that flow out of sprint planning arent registered within our Jira.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/our-journey-into-visual-scrum/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eA journey into visual Scrum\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eIn this post, I will share the experiments we conducted during the last three sprints with our physical Scrum board. The focus was about creating a better understandable visual representation of our sprint progress. Sprints are two weeks increments for us at this moment.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/scrum-sprint-misinterpretations-within-scrum/bookmarks/02-icon_hu_9b466dd874965fcb.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco Loup.\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/scrum-sprint-misinterpretations-within-scrum/bookmarks/02-thumb_hu_876db5afb4ecba2.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"do-not-change-that-name\"\u003eDo not change that name!\u003c/h2\u003e\n\u003cp\u003eThe word \u0026ldquo;Sprint\u0026rdquo; itself isn\u0026rsquo;t a problem to me actually. But the word does bring things to my mind I do not like in our Sprints.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe dont want individuality in our Sprints.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe dont want to cut corners in Sprints.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe dont want to finish first at the cost of other aspects in Sprints.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eWe need to keep a keen eye on refining our DOD and using the right kind of Acceptance Criteria for our issues to keep the quality high. And most important define a proper Sprint goal every sprint and make it physical. Another name for Sprints isn\u0026rsquo;t gonna change that. We as a team will need to make it happen to instill quality over speed.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eChanging a word like \u0026ldquo;Sprint\u0026rdquo; wont mean anything if you dont foster an environment in which the words rush and haste aren\u0026rsquo;t words that describe the work ethics. A sprint is just about an x week development cycle everyone understands, it shouldn\u0026rsquo;t be at all related to haste and rush.\u003c/p\u003e\n\u003c/blockquote\u003e\n","date_modified":"2024-02-16T08:45:23+01:00","date_published":"2023-08-07T08:20:00+02:00","id":"https://www.allsubjectsmatter.nl/scrum-sprint-misinterpretations-within-scrum/","language":"en","summary":"Changing a word like \"Sprint\" wont mean anything if you dont foster an environment in which the words rush and haste aren't words that describe the work ethics. A sprint is just about an x week development cycle everyone understands, it shouldn't be at all related to haste and rush.","tags":["Teams"],"title":"The misinterpretation of the word “Sprint” within SCRUM","url":"https://www.allsubjectsmatter.nl/scrum-sprint-misinterpretations-within-scrum/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eI am already for a while intrigued by transforming a business into a DAO (Decentralized autonomous organisation). Thinking of my role as a Product Owner i thought of a way that can work very well when transforming a part of our business into a DAO. We can benefit from Scrum, Agile and leveraging a DAO altogether this way.\u003c/p\u003e\n\u003cp\u003eIn my personal case below, a hybrid approach is preferable, where certain aspects of our business are managed by a DAO, while the rest remains under traditional management for now. My DAO would be used to manage our Front/Mid/Backoffice system called BOSS to make decisions of what to develop next, while the broader business continues to operate in a more conventional manner. This approach allows our business to benefit from the advantages of a DAO, while mitigating some of the risks and challenges.\u003c/p\u003e\n\u003cp\u003eOur goal is to develop this system into a SaaS product within the travel industry. And use a DAO for example to get our stakeholders involved. But you will read more about my idea if you carry on.\u003c/p\u003e\n\u003ch2 id=\"what-is-a-dao\"\u003eWhat is a DAO?\u003c/h2\u003e\n\u003cp\u003ea DAO, or Decentralized Autonomous Organization, is a type of organization represented by rules encoded as a computer program (called a smart contract) that is transparent, controlled by the organization members, and not influenced by a central government.\u003c/p\u003e\n\u003cp\u003eA DAO\u0026rsquo;s financial transactions and rules are maintained on a blockchain, which is a type of distributed ledger. This makes it completely autonomous and independent from any external or centralized control.\u003c/p\u003e\n\u003cp\u003eImagine it like a digital company, where all the rules and regulations are coded into a smart contract. For instance, let\u0026rsquo;s say there is a rule in a DAO that a certain proposal requires 60% of votes to pass. This rule is coded into the smart contract, and the blockchain will ensure that this rule is strictly followed, and can\u0026rsquo;t be manipulated or bypassed by anyone.\u003c/p\u003e\n\u003cp\u003eMembers of a DAO can make decisions about the rules or direction of the organization by proposing and voting on proposals, each having a vote proportional to their stake or shares in the DAO. In this way, a DAO is a bit like a traditional company but managed in a fully democratic and decentralized way.\u003c/p\u003e\n\u003cp\u003eTo sum it up, a DAO leverages blockchain technology to ensure that the organization is governed in a completely transparent, democratic and decentralized way, free from any central or external control.\u003c/p\u003e\n\u003ch2 id=\"tokens\"\u003eTokens\u003c/h2\u003e\n\u003cp\u003eCreating a unique token for a Decentralized Autonomous Organization (DAO) can be beneficial, and is a common practice, whether it\u0026rsquo;s the \u0026ldquo;best\u0026rdquo; practice depends on the specific goals and context of the DAO. Here\u0026rsquo;s why DAOs often issue their own tokens:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eVoting Rights\u003c/strong\u003e: Tokens can be used to represent voting power within the DAO. The more tokens a member holds, the more influence they have over the DAO\u0026rsquo;s decisions.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eIncentives\u003c/strong\u003e: Tokens can be used to incentivize certain behaviors or contributions to the DAO. For example, tokens can be given as rewards for completing tasks that benefit the DAO, such as coding, designing, or managing projects.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eFundraising\u003c/strong\u003e: Issuing tokens can be a method of raising funds. Potential participants can purchase tokens, effectively becoming investors in the DAO.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eCommunity Building\u003c/strong\u003e: Having a unique token can help to build a sense of community among members. It can foster a feeling of shared ownership and mutual investment in the DAO\u0026rsquo;s success.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eHowever, there are also challenges and considerations to keep in mind:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eRegulatory Issues\u003c/strong\u003e: Depending on the jurisdiction, issuing a token might be considered as issuing a security, which can have regulatory implications and legal requirements.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eToken Distribution\u003c/strong\u003e: Deciding on how to initially distribute the tokens can be challenging and controversial. The distribution method needs to be fair and align with the DAO\u0026rsquo;s goals and values.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eToken Value Fluctuation\u003c/strong\u003e: If the token is traded on the open market, its value can fluctuate, which can lead to instability and potential manipulation.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eComplexity\u003c/strong\u003e: Issuing and managing a token adds complexity to running the DAO. Not every DAO might need its own token, especially if its operations are simple or if it can effectively use an existing token.\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"how-to-leverage-a-dao-as-a-product-owner\"\u003eHow to leverage a DAO as a Product Owner\u003c/h2\u003e\n\u003cp\u003eImagine running a Decentralized Autonomous Organization (DAO) where your customers or interested parties are able to invest directly in your product. They\u0026rsquo;re granted stakes or tokens, which gives them a voice in your development process which is run in a Scrum/Agile way.\u003c/p\u003e\n\u003cp\u003eLet\u0026rsquo;s say these potential customers want a specific function, X or Y, added to your product. They can cast votes for that function using their tokens, directly influencing the development priorities. This voting mechanism can draw potential customers closer to your brand and empower existing customers to shape the product they use regularly.\u003c/p\u003e\n\u003cp\u003eIn effect, the features that top their wishlist become the focal point of your development strategy. The beauty of this approach lies in its transparency. By allowing your users to vote on potential additions to your product, you\u0026rsquo;re showing them why certain features are prioritized over others. It\u0026rsquo;s a democracy where voting requires knowing the \u0026ldquo;why\u0026rdquo; behind each decision, leading to a detailed understanding of product evolution.\u003c/p\u003e\n\u003cp\u003eThis approach is a boon not only for Product Owners but also for your business at large. Instead of cloaking your development process in secrecy, you\u0026rsquo;re inviting your users to be part of it. This participative approach can foster faster trust-building among customers and potential users than traditional methods.\u003c/p\u003e\n\u003cp\u003eAs a Product Owner, you\u0026rsquo;re still essential in filtering and interpreting customer wishes, converting them into comprehensible stories that both technical and non-technical folks can grasp.\u003c/p\u003e\n\u003cp\u003eWhen it comes time to select top priorities, your roadmap is already outlined by the democratic vote. This interaction allows you to engage and learn from your customer base quicker than relying on personas or other traditional tools to decide what comes next. Majority votes could define your sprint goals, guiding your focus for the next development cycle.\u003c/p\u003e\n\u003cp\u003eWhat better way to foster transparency and customer input using Agile/Scrum in this way? This process integrates the principles of adapt, inspect, and learn in a tightly-knit manner. With a DAO-like approach, your product evolution truly becomes a community-driven endeavor.\u003c/p\u003e\n\u003ch2 id=\"caveats\"\u003eCaveats\u003c/h2\u003e\n\u003cp\u003eThere are also caveats of course.\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eGovernance Challenges\u003c/strong\u003e: Implementing a DAO\u0026rsquo;s governance model can be complex. Balancing the interests of all stakeholders in a fair and equitable manner is a significant challenge. It\u0026rsquo;s not always the case that the majority\u0026rsquo;s decision aligns with the best interest of the project or all the stakeholders.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSmart Contract Security\u003c/strong\u003e: DAOs operate on the basis of smart contracts, which are immutable once deployed. If there\u0026rsquo;s a bug or vulnerability in the contract, it can lead to severe issues, including loss of funds. The infamous \u0026ldquo;The DAO\u0026rdquo; hack of 2016, where $50 million was siphoned due to a smart contract loophole, is an example.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eRegulatory Uncertainty\u003c/strong\u003e: DAOs exist in a legal gray area in many jurisdictions. How they\u0026rsquo;re classified and regulated by law is still uncertain in many parts of the world.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eToken Distribution\u003c/strong\u003e: How tokens are initially distributed can also be a complex issue. If not done properly, it can lead to centralization or imbalance of power in the DAO.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eLow Participation Rates\u003c/strong\u003e: DAOs rely on active participation from token holders. However, voter apathy can be a significant issue, leading to decisions being made by a small subset of stakeholders.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eShort-Termism\u003c/strong\u003e: There\u0026rsquo;s a risk that token holders who are only interested in short-term profit might vote for proposals that yield immediate gains but are detrimental in the long run.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eInterface between DAO and the Real World\u003c/strong\u003e: DAOs are digital, but their decisions often have implications in the real world, where enforcement can be more complex.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eEach of these challenges is surmountable, but they do require careful thought and design to address effectively. As with any system, the benefits should always be weighed against the potential risks and drawbacks.\u003c/p\u003e\n\u003ch2 id=\"the-role-of-a-product-owner\"\u003eThe role of a Product Owner\u003c/h2\u003e\n\u003cp\u003eThis is where a good Product Owner comes into play and lets people not vote on your whole backlog but on clearly selected items that are already enough refined and ready to be developed to a degree of uncertainty. You are still supposed to know the overall market so you can preselect the items to be worked on and let people vote on them.\u003c/p\u003e\n\u003cp\u003eYou still have stakeholders to keep happy and develop functions the market wants the most. Only by using a DAO and keeping people engaged you can adapt and learn your market way better then in other ways.\u003c/p\u003e\n\u003cp\u003eEven though a DAO is fundamentally democratic, it needs someone with a comprehensive understanding of the product to guide its development. A PO can offer insight into the feasibility and impact of features, helping to steer decision-making and prioritization.\u003c/p\u003e\n\u003ch2 id=\"smart-contracts\"\u003eSmart Contracts\u003c/h2\u003e\n\u003cp\u003eA smart contract is a computer protocol that facilitates, verifies, or enforces the negotiation or performance of a digital contract. Smart contracts run on the blockchain, making them transparent, traceable, and irreversible. They\u0026rsquo;re designed to automatically execute transactions or other specific actions when pre-set conditions are met.\u003c/p\u003e\n\u003cp\u003eHere are some key reasons why it\u0026rsquo;s important to carefully design and implement smart contracts:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eImmutability\u003c/strong\u003e: Once a smart contract is deployed on the blockchain, it cannot be altered or corrected. This immutability makes it crucial that a smart contract is thoroughly tested and reviewed before deployment to ensure that it operates as intended.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSecurity\u003c/strong\u003e: Smart contracts are vulnerable to bugs and exploits if they aren\u0026rsquo;t correctly programmed. Given the value often transacted or managed through smart contracts, security is of the utmost importance. Exploits can lead to substantial losses, as seen in the infamous DAO hack in 2016 where around $50 million was lost due to a bug in the DAO\u0026rsquo;s smart contract.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eFinancial Implications\u003c/strong\u003e: Smart contracts often manage, store, or facilitate the transfer of cryptocurrency, and errors or vulnerabilities in the contract could have significant financial repercussions.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eTrust and Reliability\u003c/strong\u003e: The trust in a smart contract, and by extension, the entire operation or system it supports, heavily relies on the contract functioning as expected. Issues could compromise the perceived reliability of the system and discourage participation or usage.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eComplexity and Interoperability\u003c/strong\u003e: Smart contracts often interact with other contracts or systems. It\u0026rsquo;s important to think through these interactions to prevent unexpected behavior. Misunderstanding or neglecting the complexities of such interactions can lead to detrimental consequences.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eRegulatory Compliance\u003c/strong\u003e: Depending on the jurisdiction and the specific use case, smart contracts need to be designed to comply with relevant laws and regulations, adding another layer of complexity to their design and implementation.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eTherefore, the design, implementation, and review of smart contracts need to be taken very seriously and conducted with utmost diligence. In many cases, it\u0026rsquo;s advisable to have smart contracts audited by a specialized security firm before they are deployed.\u003c/p\u003e\n\u003ch2 id=\"a-written-sample-of-a-smart-contract\"\u003eA written sample of a Smart Contract\u003c/h2\u003e\n\u003cp\u003eSo you need a good working contract on beforehand. The Product Owner can be the manager of such a smart contract for example. He adds the issues to the system. and the stakeholders can vote on these issues using their tokens.\u003c/p\u003e\n\u003cp\u003eEach vote raises the importance of the issues to be developed. After which you pick the highest ranked issues to develop on while keeping an eye of what you can work in a single sprint ofcourse.\u003c/p\u003e\n\u003cp\u003eYou can also use two types of tokens for example. Normal tokens for your customers using your product and social tokens for potential customers for example.\u003c/p\u003e\n\u003cp\u003eValidity of tokens is also something to consider since wishes can become obsolete after time.\u003c/p\u003e\n\u003cp\u003eThis all can become quite complicated and i might post a full sample of a smart contract written in Solidity in a later blog using the conditions above.\u003c/p\u003e\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eThis has become a somewhat longer article then usual but it needed a lot of explanation to be able to understand what it means to leverage your role as a PO inside a DAO.\u003c/p\u003e\n\u003cp\u003eI hope this has helped a bit to clarify and explain what a DAO is and how it can be used especially from a SCRUM/Agile Product Owner point of view where both can really thrive.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://www.scrum.org/francois-fort\" data-external\u003eFrancois Fort\u003c/a\u003e wrote a article about this on \u003ca href=\"https://www.scrum.org/resources/blog/are-daos-future-agile-organizations\" data-external\u003escrum.org\u003c/a\u003e Are DAOs the future of Agile Organisations.\u003c/p\u003e\n\u003cp\u003eYou need stakeholders to buy in and what is better then to have engaged stakeholders with stakes in your product leveraging a DAO? You get detailed and immediate responses from the market when you have a engaged diversified group of people wanting your product to succeed because they then also succeed .\u003c/p\u003e\n","date_modified":"2025-01-07T13:36:33+01:00","date_published":"2023-07-25T15:37:00+02:00","id":"https://www.allsubjectsmatter.nl/dao-and-agile-from-traditional-businesses-to-a-dao-creates-opportunities-for-product-owners/","language":"en","summary":"I am already for a while intrigued by transforming a business into a DAO (Decentralized autonomous organisation). Thinking of my role as a Product Owner i thought of a way that can work very well when transforming a part of our business into a DAO","tags":["Strategy","Teams"],"title":"From Traditional Businesses to DAOs: Opportunities for Product Owners","url":"https://www.allsubjectsmatter.nl/dao-and-agile-from-traditional-businesses-to-a-dao-creates-opportunities-for-product-owners/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003cp\u003eSomething totally not related to Agile or Tech but a little insight about me having found inline skating again and some history with inline skating.\u003c/p\u003e\n\u003ch2 id=\"the-past\"\u003eThe past\u003c/h2\u003e\n\u003cp\u003eRollerblading has been a big part of my life in the past, as intrinsic to my childhood as the summers I spent with skinned knees and bruised elbows. In the early-90s, when I was about 8 or 9, I started inline skating since everybody had skates back then, a hobby that quickly turned into a passion. For me, the rattle of wheels on pavement was a symphony, a celebration of youth, freedom, and daring if i look back at it.\u003c/p\u003e\n\u003cp\u003eBack then, halfpipe skating was the major thing i did when i was about 12. The adrenaline, the thrill of conquering gravity, it was addictive. I spent countless hours on that halfpipe, the metal under my wheels becoming as familiar as the back of my hand. It wasn\u0026rsquo;t just about nailing a trick, it was about the rush, the exhilarating blend of fear and anticipation that hit just before you pushed off the ramp.\u003c/p\u003e\n\u003cp\u003eAway from the halfpipe, I\u0026rsquo;d also dive into the world of streethockey. Decked out in kneepads, and my trusty Roces streets strapped on, I was ready to overcome anything that came my way. Chasing that ball across the tarmac was the best kind of chaos, and i loved every minute of it.\u003c/p\u003e\n\u003cp\u003eBy the time I was 16, my passion for inline skating somehow started to fade. I don\u0026rsquo;t remember why I stopped – perhaps life got in the way, or I just got distracted by other pursuits. However, the flame was never fully extinguished, just simmering on the surface waiting for a chance to ignite again.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/inline-skating-these-days/images/halfpipesept1998_hu_950a645dec9922f6.webp\" alt=\"\" width=\"560\" height=\"745\" data-zoom-src=\"https://www.allsubjectsmatter.nl/inline-skating-these-days/images/halfpipesept1998_hu_12b1ec34bdac5bf.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eMe in the back with the light brown trousers (14y) september of 1998 (Roces Streets)\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"the-present\"\u003eThe present\u003c/h2\u003e\n\u003cp\u003eThis chance came in early 2022, when my children showed an interest in inline skating. Their curiosity rekindled my own passion, and I found myself asking: why did I ever stop? It was then that I decided to purchase the skates i always wanted as a little kid and enter the world I had left behind again.\u003c/p\u003e\n\u003cp\u003eAfter a visit to the \u0026lsquo;\u003ca href=\"https://www.thisissoul.com/\" data-external\u003ethisissoul\u003c/a\u003e\u0026rsquo; store, where I was assisted by incredibly helpful staff who guided me through trying on various models due to my wide feet. Yet, my mind was already set on one particular model: the Roces 5th Element, the skate of my childhood dreams. Slipping into those skates again was like stepping back into my younger self, a wonderful feeling of nostalgia and renewed excitement that i now had 5th elements.\u003c/p\u003e\n\u003cp\u003eToday, my Roces are fitted with a rockered \u0026lsquo;Wish\u0026rsquo; (\u003ca href=\"https://www.oysius.com/mc65/\" data-external\u003eOysi\u003c/a\u003e) frame, with a 0.5mm rocker, enhancing control and agility on the skates. This technical setup harks back to my halfpipe days, yet also caters to the more rhythmic, flowing style of Wizard Skating.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/inline-skating-these-days/images/20230813_191301_hu_3db90f8add70e8e2.webp\" alt=\"\" width=\"560\" height=\"262\" data-zoom-src=\"https://www.allsubjectsmatter.nl/inline-skating-these-days/images/20230813_191301_hu_d051f68e377328f.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eThe halfpipe i learned my halfpipe skating in since 1995. It is still standing strong today. 2 meters tall\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003cp\u003eLooking back, my journey with inline skating has truly come full circle. It\u0026rsquo;s a wonderful feeling to share this passion with my children, just as I did when I was their age. The joy, the freedom, the adrenaline – they’re all still there, as vivid as they were in my younger days. Inline skating has evolved, and so have I along with it. And I wouldn\u0026rsquo;t have it any other way.\u003c/p\u003e\n\u003cp\u003ePicking up where I left off, my return to the inline skating world has been nothing short of invigorating. Each skate session gives me a sense of joy, an eagerness to rekindle old skills and learn new ones. With my trusty Roces strapped on, it just felt like riding a bike. Once learned you never unlearn.\u003c/p\u003e\n\u003cp\u003eThe best part of this journey has been sharing it with my children. Seeing them strap on their skates and navigate the world on wheels, much like I did at their age, fills me with a sense of nostalgia and pride. Their squeals of joy when they master a new skill, their determination when they stumble and get back up, it\u0026rsquo;s a beautiful reminder of why I fell in love with this sport in the first place.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/inline-skating-these-days/images/IMG20230513141741_hu_1d193cd4f4f394a5.webp\" alt=\"\" width=\"560\" height=\"420\" data-zoom-src=\"https://www.allsubjectsmatter.nl/inline-skating-these-days/images/IMG20230513141741_hu_55753694c7e82f93.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eMe, and my 2 little girls on inline skates.\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"nostalgia\"\u003eNostalgia\u003c/h2\u003e\n\u003cp\u003eIt\u0026rsquo;s strange to think about why I ever left inline skating. Maybe I needed that break to appreciate what I had missed truly. Regardless, I’m grateful for the chance to return to my roots and re-discover my love for this dynamic sport.\u003c/p\u003e\n\u003cp\u003eAnd so, as I glide through the streets, the wind in my face and the world rushing by, I can\u0026rsquo;t help but feel like that 10-year-old kid again, just starting on his rollerblading journey. Only now, I have the added pleasure of sharing this passion with my kids.\u003c/p\u003e\n\u003cp\u003eSo here\u0026rsquo;s to the Roces 5th Elements, to the rockered \u0026lsquo;Wish\u0026rsquo; (Oysi) frame, to the adrenaline of the halfpipe, and the smooth flow of Wizard Skating. Here\u0026rsquo;s to the joy of falling and getting back up, and most importantly, to the sheer love of rollerblading. Because, at the end of the day, it’s not just about the skates, the style, or the tricks. It\u0026rsquo;s about the ride, the freedom, and the lifelong memories that come with it.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/inline-skating-these-days/images/skating-sept-1995_hu_88c894b8717da63f.webp\" alt=\"\" width=\"538\" height=\"355\" data-zoom-src=\"https://www.allsubjectsmatter.nl/inline-skating-these-days/images/skating-sept-1995_hu_88c894b8717da63f.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eLittle me at the age of 11. Cruising around on my Rollerblade Zetrablades.\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"utrecht-skateparade\"\u003eUtrecht Skateparade\u003c/h2\u003e\n\u003cp\u003eOne of the coolest things of my return to inline skating has also been participating in the \u003ca href=\"https://u-skateparade.nl/home/\" data-external\u003eUtrecht Skate Parade\u003c/a\u003e. The event is a celebration of skating culture, bringing together rollerbladers of all ages and skill levels from across the city and beyond. It\u0026rsquo;s a vibrant display of community, all on wheels skating a tour of around 25km.\u003c/p\u003e\n\u003cp\u003eThis Parade is held every week from may till the end of september. We take the car roads through the heart of the City Utrecht. With every week a different DJ installed on the vehicle in the front of the Parade.\u003c/p\u003e\n\u003cp\u003eThere\u0026rsquo;s a unique magic to the parade. The feeling of skating through the streets of Utrecht, surrounded by fellow enthusiasts with some nice beats to listen to, is hard to put into words. To me the best feeling to describe it is a very chill festival vibe.\u003c/p\u003e\n\u003cp\u003eI try to attend as many parades as I can. Every time, I’m met with the same exhilarating energy. I can\u0026rsquo;t help but be swept up in the collective spirit of hundreds of skaters, all moving together. It’s a powerful reminder of the joy that inline skating can bring.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eIt\u0026rsquo;s not just a sport, it\u0026rsquo;s a lifestyle.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI can\u0026rsquo;t wait for my next skate. These are the moments that make my inline skating journey worthwhile. And each parade, each glide through the city streets, brings me closer to the core of what inline skating means to me – freedom, joy, and a sense of boundless possibility.\u003c/p\u003e\n\u003cfigure class=\"embed\"\u003e\u003ciframe src=\"https://www.youtube-nocookie.com/embed/s4xHK8CHRcQ\" title=\"Skateparade 20 augustus door Utrecht\" loading=\"lazy\" allow=\"accelerometer; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen referrerpolicy=\"strict-origin-when-cross-origin\"\u003e\u003c/iframe\u003e\u003c/figure\u003e\n\n\u003cfigure class=\"embed\"\u003e\u003ciframe src=\"https://www.youtube-nocookie.com/embed/ovyFXJ5YkM0\" title=\"Aktiesport | Summer Event | Utrecht Skate Parade\" loading=\"lazy\" allow=\"accelerometer; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen referrerpolicy=\"strict-origin-when-cross-origin\"\u003e\u003c/iframe\u003e\u003c/figure\u003e\n\n","date_modified":"2023-10-11T10:32:32+02:00","date_published":"2023-06-23T14:08:23+02:00","id":"https://www.allsubjectsmatter.nl/inline-skating-these-days/","language":"en","summary":"Something totally not related to Agile or Tech but an little story how i feel about having found inline skating again and my past with inline skating.","tags":["Inline skating"],"title":"Inline skating these days","url":"https://www.allsubjectsmatter.nl/inline-skating-these-days/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eWithout a \u0026ldquo;good\u0026rdquo; SME (Subject Matters Expert) stance, Every product is a lesser product then it can be.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThis is the bomb I am am dropping here to start with.\u003c/p\u003e\n\u003cp\u003eThere are off course nuances to this statement. I will explain below why this fact is true in my opinion. When you are a SME you care for your product in my opinion. Why would you know so much about the product otherwise? And also have all this information available? I tend to forget things i dont care about\u0026hellip;. So i must care about our product.\u003c/p\u003e\n\u003ch2 id=\"finding-the-nuance-within-a-sme\"\u003e\u003cstrong\u003eFinding the nuance within a SME\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003eThe struggle is that you need to teach yourself how to react when you are uncertain or wrong. Being stubborn because you are afraid to fail is the wrong take on this stance. You also need to give other people the chance to learn what you might already know. Maybe even some new insights come to the surface! Dictating an answer isn\u0026rsquo;t going to help in accepting your answer until they figure it out themselves. And even more importantly it wont make the team smarter if they dont figure things out themself. The whole team needs to benefit, not just you.\u003c/p\u003e\n\u003cp\u003eThis requires tact and the right moment to step in to certain discussions. You dont want to lead such a discussion but steer it in a natural way to the right direction when it occurs. And do remember there is still always the possibility that you are indeed wrong!\u003c/p\u003e\n\u003ch2 id=\"a-good-subject-matter-expert\"\u003eA \u0026ldquo;good\u0026rdquo; Subject Matter Expert\u003c/h2\u003e\n\u003cp\u003eBeing a SME is bloody darn handy when you have the upper hand on the positives a Subject Matter Expert can bring. That\u0026rsquo;s why this is one the most interesting misunderstood stances of a Product Owner to me. You can be totally in the wrong (misunderstood) as a SME or totally in the right. In a positive way without bothering yourself with everything, every time, again. And that is where you need to find your nuance within your role as a Product Owner with a SME mindset. Also if you know the details of something, also explain the WHY to the team. Keep involving them instead of being Mr Smart-ass.\u003c/p\u003e\n\u003ch2 id=\"limitations\"\u003eLimitations\u003c/h2\u003e\n\u003cp\u003eIt is all a matter of knowing your limitations and being honest about it. You know a lot but not everything and the world evolves so things might be different nowadays. For me this involves in keeping my mouth shut when i am not sure. And also sometimes i give my take on why we made a choice in the past but i encourage everyone to seek it out in the present day since the problem back then might not be a problem anymore. And you always learn from these things even though the outcome might be the same.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eIt is good practice if there is somebody to give context to the choices made in the past without steering into a certain direction. I think there is a lot of value in explanations this way.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"an-analogy\"\u003e\u003cstrong\u003eAn analogy\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003eThe product we developed as a team started as a child of me. The vision came from me. It took us 2 years before we gave labor. And after that 2 year labor we teamed up to raise our child in the best possible way. This gave a lot of headaches but also brought us so much in learning and adapting to each other. We all started to care for this child.\u003c/p\u003e\n\u003cp\u003eHow can i not pretend to be a SME for such a product? I am the one who started the initial idea. I was there when we gave labor after a 2 year period of close development. I was there all the way through its childhood watching it grow up and get mature. And even though it might be mature now our child still needs guidance and help from time to time. It is still my child after all. That more and more people also started to really care for this child changes nothing to the feeling i have towards our product.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eSpread the love for your product across your team. Dont keep it to yourself. Make it better as a team.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"conclusion\"\u003e\u003cstrong\u003eConclusion\u003c/strong\u003e\u003c/h2\u003e\n\u003cp\u003eI cant help it that i still remember almost every choice we made for our product in the past. And also why we made this choice within our development cycles. I dont particular choose to be that wise guy but this is a topic of interest for me and therefore i remember. I do need to give other people room for their own thoughts tho. So yeah i can relate to, and totally find myself within a Subject Matter Expert to a certain extend.\u003c/p\u003e\n\u003cp\u003eYou do need to know/feel WHEN to step into a discussion. Dont get in a discussion being the wise guy all the time. This all might sound cocky or arrogant. But we all want the same and that is delivering a better product then it already was. If we keep talking in a respectful way without acting as the wise guy we are on the right track as a team.\u003c/p\u003e\n\u003cp\u003eAnd again! keep explaining the WHY\u0026rsquo;s and speak your mind. involve your team in decisions and explanations of choices made in the past. You are a team together with all people not just as the PO/SME.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eDisclaimer: despite highlighting one stance of the Scrum master stances here. It is a combination of all stances that make you a better Product Owner. This is just one stance that interests me for being a misunderstood stance (and for a good reason). Without a clear Vision and \u0026ldquo;being\u0026rdquo; all the other stances you are going to have a hard time as a Product Owner.\u003c/em\u003e\u003c/p\u003e\n","date_modified":"2023-05-03T12:17:32+02:00","date_published":"2023-04-28T13:56:58+02:00","id":"https://www.allsubjectsmatter.nl/scrum-a-understood-subject-matter-expert-sme-misunderstood-stances/","language":"en","summary":"Without a \"good\" SME (Subject Matter Expert) stance within SCRUM, every product is a lesser product then it can be. That's why this is one the most interesting misunderstood stances of a Product Owner to me.","tags":["Teams"],"title":"A understood Subject Matter Expert","url":"https://www.allsubjectsmatter.nl/scrum-a-understood-subject-matter-expert-sme-misunderstood-stances/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eIn this post, I will share the experiments we conducted during the last three sprints with our physical Scrum board. The focus was about creating a better understandable visual representation of our sprint progress. Sprints are two weeks increments for us at this moment.\u003c/p\u003e\n\u003ch2 id=\"our-scrum-board-right-now\"\u003eOur Scrum board right now.\u003c/h2\u003e\n\u003cp\u003eThe image I will post below displays the board we have been using for almost a year. As we began to take Scrum more seriously we started to notice a lack of transparency with this board.\u003c/p\u003e\n\u003cp\u003eIn this board we already resolved many of the problems we had previously encountered. These improvements are explained in detail in a previous post: \u003ca href=\"https://www.allsubjectsmatter.nl/sprints-in-real-life/\" data-preview\u003eOur sprints in real life\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eTo gain a better understanding of our journey, \u0026ldquo;\u003ca href=\"https://www.allsubjectsmatter.nl/achieving-flow-with-scrum/\" data-preview\u003eAchieving FLOW with Scrum\u003c/a\u003e\u0026rdquo; is also a must-read. It delves into our background and, more importantly, answers the question: Why are we doing this?\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/our-journey-into-visual-scrum/images/20230307_125843-2_hu_4a63e1b954b7439e.webp\" alt=\"\" width=\"560\" height=\"632\" data-zoom-src=\"https://www.allsubjectsmatter.nl/our-journey-into-visual-scrum/images/20230307_125843-2_hu_6af505125fa866e7.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eStage 1 before starting a new experimentation phase\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003cp\u003eThese are the lanes and setup we use at this moment. We have split up the last two (Testing and Done) to improve transparency within our current CI/CD setup. Below is a list of things we have already tested and therefore are implemented on this board. On these improvements we will continue to incorporate when experimenting with new versions of our board.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eThe sprint goal is clearly visible.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe use green colored cards for the items contributing to achieving the sprint goal.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe mark tasks that are in testing to long with a visual exclamation mark.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe use \u0026ldquo;released\u0026rdquo; and \u0026ldquo;not yet released\u0026rdquo; in our Done lane to keep things that are DOD but not yet deployed visible for the Scrum team.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eOur work order is put on the board in the Todo lane from top to bottom. Top items get worked on first unless there appears a good reason not to work in that order :)\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eOther tasks we committed to for the sprint are marked blue.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eRandom bugfixes and potential hotfixes uncovered during a sprint are marked red. They are put in the bottom of the board. They may be important but they came up during a sprint and are visible in the bottom of the \u0026ldquo;TODO\u0026rdquo; lane therefore.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eSubtasks are always defined on this board. They are for the developers to do the work to deliver the full story. They may reside in Jira but the physical board is leading.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWe use name cards to show who is working on what.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe reason we started experimenting with different board setups you will find below is that we increasingly started breaking stories down into subtask. However, when our developers completed a subtask. We didn\u0026rsquo;t want to remove it from the board immediately for a whole lof ot different reasons.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eRemoving a subtask creates less transparency and instilled less talks about done subtasks.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eIt gives developers that were out of the office (free) insight in tasks done when they come into the office the next day.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eThe most important topic was the lack of transparency this board still had on subtasks!\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eIt also feels more satisfying to move a completed subtask to a designated area on the board. Slap it there, and move on. This provides a sense of closure. A feeling you wont get when you immediately remove, clean, and wipe out what you just accomplished.\u003c/p\u003e\n\u003cp\u003eIn the first version of our board we didn\u0026rsquo;t account for this. Sometimes subtasks would move with the story. While other times a subtask moved to the \u0026ldquo;Done\u0026rdquo; lane. This is wrong! there isn\u0026rsquo;t anything to release or not yet released. It is a completed subtask of a branch probably not even committed yet. This situation created a lot of issues with regards to transparency.\u003c/p\u003e\n\u003ch2 id=\"first-iteration-changes\"\u003eFirst iteration changes\u003c/h2\u003e\n\u003cp\u003eWe started to experiment with splitting up the \u0026ldquo;Dev\u0026rdquo; lane with a \u0026ldquo;Doing\u0026rdquo; and \u0026ldquo;Done\u0026rdquo;. This way we could keep the completed subtasks in a designated lane. Although we noticed some initial irritation in starting with this setup. It was essential to try it out though to determine its effectiveness!\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/our-journey-into-visual-scrum/images/20230314_082856_hu_2559b27601cc5669.webp\" alt=\"\" width=\"560\" height=\"640\" data-zoom-src=\"https://www.allsubjectsmatter.nl/our-journey-into-visual-scrum/images/20230314_082856_hu_d520949d7c9945da.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eStage 2. Experimenting with a doing/done lane in Development.\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"first-iteration-conclusion\"\u003eFirst iteration conclusion\u003c/h2\u003e\n\u003cp\u003eAfter using this setup for two weeks, our final conclusion is that it\u0026rsquo;s helpful to have a designated area for placing \u0026ldquo;Done\u0026rdquo; subtasks. However, including this area within the current lanes of the board is not ideal. Completed subtasks should no longer be visible like this, they now appear as if they still need to move further to the right.\u003c/p\u003e\n\u003cp\u003eSubtasks themselves don\u0026rsquo;t move in Testing for example, as they are part of a story that holds the value. It is the story that progresses into Review, Testing, and Done. Thus, this approach is not recommended in our opinion.\u003c/p\u003e\n\u003cp\u003eWe did agree to remove the cards when the developers as a collective felt it was time to remove them. But this method also doesn\u0026rsquo;t provide transparency either since there is no clear rule in place.\u003c/p\u003e\n\u003ch2 id=\"second-iteration-changes\"\u003eSecond iteration changes\u003c/h2\u003e\n\u003cp\u003eTogether we came up with another version of the board. This version is again visible in the image below.\u003c/p\u003e\n\u003cp\u003eWe reverted back to the lanes we had in the start of this experiment. Todo, Dev, Review, Testing and Done. We removed the visual representation of Todo and placed the Sprint goal there instead. We all agreed this was a more suitable location for the Sprint Goal.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/our-journey-into-visual-scrum/images/20230329_121747_hu_733fc66d19cc2329.webp\" alt=\"\" width=\"560\" height=\"1093\" data-zoom-src=\"https://www.allsubjectsmatter.nl/our-journey-into-visual-scrum/images/20230329_121747_hu_93038d66e3d24fd1.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eStage 3. Bringing down the number of lanes and introduce a \"Burn pile\" with all done subtasks.\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003cp\u003eWe also decided never to move any work above the items representing the achievement of our sprint goal. For instance, if we start working on a blue or red item. This item moves to \u0026ldquo;Dev\u0026rdquo; while staying on the same horizontal line where it was initially placed. So, from now on, we aim to keep items horizontally aligned as they move through the lanes. The only exception is the Done lane, which is divided into sections for released and unreleased work.\u003c/p\u003e\n\u003cp\u003eThe \u0026ldquo;Burn Pile\u0026rdquo; is a name we came up with for all done subtasks. We wanted to avoid having two \u0026ldquo;Done\u0026rdquo; lanes. It just feels satisfying to place the subtasks you just finished in this designated area! This approach not only provides a sense of closure but also offers insight into everything completed as subtasks.\u003c/p\u003e\n\u003ch2 id=\"second-iteration-conclusion\"\u003eSecond iteration conclusion\u003c/h2\u003e\n\u003cp\u003eThis is the first iteration of our \u0026ldquo;Burn Pile\u0026rdquo; version. We have already discussed some other considerations for trying out this version.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eWe might need to add a reference to the story for transparency reasons. Or group the same subtasks of a story together for example.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eWe will learn from this experience by simply putting it into practice. It\u0026rsquo;s essential not to overcomplicate things immediately when it might not be necessary. As seen in the screenshot, we have already placed some subtasks in the Burn Pile, and we all agree that it feels good to physically do so.\u003c/p\u003e\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eWe have decided that this board version is what we will be moving forward with for now. The first iteration did not work well due to having too many lanes, too much space, two Done lanes, and subtasks remaining too visible for too long after completion.\u003c/p\u003e\n\u003cp\u003eSo moving forward from now on we will be using this version. I will keep updating this post every time we find improvements within our work using this board.\u003c/p\u003e\n","date_modified":"2023-04-11T14:09:23+02:00","date_published":"2023-03-29T17:16:59+02:00","id":"https://www.allsubjectsmatter.nl/our-journey-into-visual-scrum/","language":"en","summary":"In this post, I will share the experiments we conducted during the last three sprints with our physical Scrum board. The focus was about creating a better understandable visual representation of our sprint progress. Sprints are two weeks increments for us at this moment.","tags":["Teams"],"title":"A journey into visual Scrum","url":"https://www.allsubjectsmatter.nl/our-journey-into-visual-scrum/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eI am writing this introduction because i want to explain why I use ChatGPT and what i use it for. At first i was a bit sceptical of what ChatGPT was. It was to me more of a fantast.\u003c/p\u003e\n\u003cp\u003eBut that is only when you ask questions you have no deep knowledge about. We have to be very careful there since you dont know when it is making stuff up. And also asking the right kind of questions to get a decent enough answer helps ChatGPT not making things up themself.\u003c/p\u003e\n\u003ch2 id=\"practical-examples\"\u003ePractical examples\u003c/h2\u003e\n\u003cp\u003eIn our work area we tested it for example to write intro texts for a hotel we had. ChatGPT made up a lot of stuff about swimming pools available and such. This particular hotels has no swimming pools.\u003c/p\u003e\n\u003cp\u003eSo it isn\u0026rsquo;t useful there. But it is actually useful about topics you do know about. You can ask it the right questions and it makes up a tekst which is for at least 75% usable most of the time. This will safe time.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eChatGPT is for language what the Calculator was for Mathematics.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eChatGPT is going to make our life a whole lot easier by providing information about topics you want to write about and already have knowledge about.\u003c/p\u003e\n\u003ch2 id=\"chatgpt-and-allsubjectsmatter\"\u003eChatGPT and AllSubjectsMatter\u003c/h2\u003e\n\u003cp\u003eHow I use ChatGPT on this website is to correct my grammar and sentence flow. I am a native Dutch person. And how i would like to brag that I used to do so well in my English classes. I never did any homework back in school but still got 9\u0026rsquo;s and 10\u0026rsquo;s.\u003c/p\u003e\n\u003cp\u003eI have to admit now that this has all become very rusty. Especially since i havent written in English for years on a regular basis until now. So it is actually helping me at this moment to write better readable English texts.\u003c/p\u003e\n\u003cp\u003eThis is what ChatGPT produced of the above lines.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eI use ChatGPT to correct my grammar and sentence flow, as I am a native Dutch speaker. I would like to brag that I used to excel in my English classes, never completing any homework but still receiving 9\u0026rsquo;s and 10\u0026rsquo;s. However, I must admit that my English has become quite rusty since I haven\u0026rsquo;t written in the language regularly for years. Currently, ChatGPT is helping me improve my ability to write more readable English texts.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eAnd this is the same text after i corrected it with \u003ca href=\"https://hemingwayapp.com/\" data-external\u003eHemingway\u003c/a\u003e\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cem\u003eI use ChatGPT to correct my grammar and sentence flow, as I am a native Dutch speaker. I would like to brag that I used to excel in my English classes.\u003c/em\u003e\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cem\u003eI never did any homework but still received 9\u0026rsquo;s and 10\u0026rsquo;s in school. But I must admit that my English has become quite rusty since I haven\u0026rsquo;t written in the language often for years. For now ChatGPT is helping me improve my ability to write more readable English texts.\u003c/em\u003e\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eThe original already isn\u0026rsquo;t to bad to me but this is just an example of what we can do with tools that help with readability issues.\u003c/p\u003e\n\u003ch2 id=\"what-is-hemingway\"\u003eWhat is Hemingway?\u003c/h2\u003e\n\u003cp\u003e\u003ca href=\"https://hemingwayapp.com/\" data-external\u003eHemingway\u003c/a\u003e is a visual readability editor which can help you to simplify sentences. I use this all the time to check all my English texts for readability. This also helps make your texts easier to read.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/chatgpt-and-the-future-of-blogging/images/image-4_hu_d84c0d75ce005095.webp\" alt=\"\" width=\"560\" height=\"295\" data-zoom-src=\"https://www.allsubjectsmatter.nl/chatgpt-and-the-future-of-blogging/images/image-4_hu_7c4c22a11578f1f7.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eI use one of these two tools to try and write better English. Sometimes i use both of them. This is the place where ChatGPT can really shine in helping you to improve your writing. By using ChatGPT I immediately started learning to write clearer and easier sentences.\u003c/p\u003e\n\u003cp\u003eSo there is hopefully less and less to correct every time i feed a text to Hemingway or ChatGPT. This is because you also learn yourself by running your texts through these tools.\u003c/p\u003e\n\u003cp\u003eI dont like using ChatGPT to fully write my posts and redirect them into something that represents \u0026ldquo;me\u0026rdquo; since that doesnt feel right to me. The idea has to come from me and i use ChatGPT to polish it sometimes. And not the other way around for me.\u003c/p\u003e\n","date_modified":"2023-06-23T14:22:45+02:00","date_published":"2023-03-22T20:09:35+01:00","id":"https://www.allsubjectsmatter.nl/chatgpt-and-the-future-of-blogging/","language":"en","summary":"I will explain how a native Dutch speaker utilizes ChatGPT and Hemingway to enhance grammar, sentence flow, and readability in my English texts, resulting in hopefully more polished and comprehensible writing.","tags":["Building"],"title":"ChatGPT and the future of blogging","url":"https://www.allsubjectsmatter.nl/chatgpt-and-the-future-of-blogging/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"update-13-11-2024\"\u003eUpdate (13-11-2024)\u003c/h2\u003e\n\u003cp\u003eSomehow i lost the creating of a new image to copy all the builder items from. It changed nothing about the way the image worked but the image size is much smaller now. It is under 100mb from more then 2gb. Somewhere in my previous debugging we lost this step. So the lastest update can be found below again.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/azure-western-europe-zone-3-latency-issue-exploration/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eLatency Anomaly: An attempt to untangle Azure Performance issues in Western Europe\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eUnveiling a significant Azure performance issue: Zone 3’s unexpected latency anomaly in the Western Europe region challenges cloud deployment strategies\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/docker-nginx-http3-and-brotli/bookmarks/01-icon_hu_9b466dd874965fcb.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco Loup.\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/docker-nginx-http3-and-brotli/bookmarks/01-thumb_hu_27e6839ff9d3a962.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eIn here you will find a working Dockerfile I created with the below defined compatibility needs for myself. This is posted here for my own archive and to share of course. I added the required support for the following.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eHTTP/2 HPACK\u003c/li\u003e\n\u003cli\u003eHTTP/3\u003c/li\u003e\n\u003cli\u003eBoringssl\u003c/li\u003e\n\u003cli\u003eBrotli support\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThis version of Nginx supports HTTP/3 quic and Brotli compression as a main benefit with some other fixes. Not having Brotli support these days is just a step back in the delorean.\u003c/p\u003e\n\u003cp\u003eMore info about how \u003ca href=\"https://blog.cloudflare.com/hpack-the-silent-killer-feature-of-http-2/\" data-external\u003eHPACK\u003c/a\u003e accelerates your HTTP/2. And here you can read more details about \u003ca href=\"https://blog.ononoki.org/enable-http3-for-nginx/\" data-external\u003eBoringSSL\u003c/a\u003e\u003c/p\u003e\n\u003ch2 id=\"brotli-compared-to-gzip\"\u003eBrotli compared to GZIP\u003c/h2\u003e\n\u003cp\u003eAs mentioned this is a Nginx version which supports Brotli. Brotli is very useful as it is a better compression algorithm then GZIP. If you are still not convinced watch the video below.\u003c/p\u003e\n\u003cfigure class=\"embed\"\u003e\u003ciframe src=\"https://www.youtube-nocookie.com/embed/BMCvMWqoivs\" title=\"How File Compression Increases Your Website Speed: BROTLI vs GZIP Algorithms\" loading=\"lazy\" allow=\"accelerometer; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen referrerpolicy=\"strict-origin-when-cross-origin\"\u003e\u003c/iframe\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"nginx-base-docker-file\"\u003eNginx base Docker file\u003c/h2\u003e\n\u003cp\u003eThis is the base of my dockerfile which takes care of all the things mentioned above. You can of course edit this or use different versions of alpine or Nginx regarding this wont break building the other modules. I also updated Nginx, NJS and BoringSSL to the latest versions.\u003c/p\u003e\n\u003cp\u003eI use this image instead of the official Nginx to load all my web projects. See this as a base image to define all your running webserver projects in. Only you get \u0026ldquo;free\u0026rdquo; Brotli compression and HTTP/3 support with it so you can make your sites faster and more up to date.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-docker\"\u003e\u003ccode class=\"language-docker\"\u003e##################################################\n# Nginx with Quiche (HTTP/3) and Brotli\n# VERSION 1.05 / 13-11-2024\n##################################################\n# This build compiles Nginx with Brotli and\n# HTTP/3 support. It uses a multi-stage build to\n# minimize the final image size by copying only\n# the necessary artifacts from the builder image.\n# Based on Alpine for a lean and efficient image.\n##################################################\n\n# Builder Stage\nFROM alpine:latest AS builder\n\n# Set the versions of Nginx, NJS, and BoringSSL\nENV NGINX_VERSION=1.27.2\nENV NJS_VERSION=0.8.7\nENV BORINGSSL_VERSION=c59bf8bf189dcbde868e04efcd53b705ed155231\nENV BORINGSSL=\u0026amp;#34;/tmp/boring-nginx\u0026amp;#34;\n\n# Build-time metadata\nARG BUILD_DATE\nARG VCS_REF\n\n# Install dependencies for building Nginx\nRUN addgroup -S nginx \\\n  \u0026amp;amp;\u0026amp;amp; adduser -D -S -h /var/cache/nginx -s /sbin/nologin -G nginx nginx \\\n  \u0026amp;amp;\u0026amp;amp; apk update \\\n  \u0026amp;amp;\u0026amp;amp; apk upgrade \\\n  \u0026amp;amp;\u0026amp;amp; apk add --no-cache ca-certificates openssl \\\n  \u0026amp;amp;\u0026amp;amp; update-ca-certificates \\\n  \u0026amp;amp;\u0026amp;amp; apk add --no-cache --virtual .build-deps \\\n      gcc libc-dev make pcre-dev zlib-dev linux-headers gnupg \\\n      libxslt-dev gd-dev geoip-dev perl-dev \\\n  \u0026amp;amp;\u0026amp;amp; apk add --no-cache --virtual .brotli-build-deps \\\n      autoconf libtool automake git g\u0026#43;\u0026#43; cmake go perl rust cargo patch \\\n      libxml2-dev byacc flex libstdc\u0026#43;\u0026#43; libmaxminddb-dev lmdb-dev file openrc\n\n# Install and build BoringSSL\nRUN mkdir $BORINGSSL \\\n  \u0026amp;amp;\u0026amp;amp; cd $BORINGSSL \\\n  \u0026amp;amp;\u0026amp;amp; git clone https://boringssl.googlesource.com/boringssl \\\n  \u0026amp;amp;\u0026amp;amp; cd boringssl \\\n  \u0026amp;amp;\u0026amp;amp; git checkout -q ${BORINGSSL_VERSION} \\\n  \u0026amp;amp;\u0026amp;amp; mkdir build \\\n  \u0026amp;amp;\u0026amp;amp; cd build \\\n  \u0026amp;amp;\u0026amp;amp; cmake -DBUILD_SHARED_LIBS=1 .. \\\n  \u0026amp;amp;\u0026amp;amp; make\n\n# Prepare BoringSSL for Nginx compilation\nRUN mkdir -p \u0026amp;#34;$BORINGSSL/boringssl/.openssl/lib\u0026amp;#34; \\\n  \u0026amp;amp;\u0026amp;amp; cd \u0026amp;#34;$BORINGSSL/boringssl/.openssl\u0026amp;#34; \\\n  \u0026amp;amp;\u0026amp;amp; ln -s ../include include \\\n  \u0026amp;amp;\u0026amp;amp; cd \u0026amp;#34;$BORINGSSL/boringssl\u0026amp;#34; \\\n  \u0026amp;amp;\u0026amp;amp; cp \u0026amp;#34;build/crypto/libcrypto.so\u0026amp;#34; \u0026amp;#34;.openssl/lib\u0026amp;#34; \\\n  \u0026amp;amp;\u0026amp;amp; cp \u0026amp;#34;build/ssl/libssl.so\u0026amp;#34; \u0026amp;#34;.openssl/lib\u0026amp;#34; \\\n  \u0026amp;amp;\u0026amp;amp; cp \u0026amp;#34;.openssl/lib/libssl.so\u0026amp;#34; /usr/lib/ \\\n  \u0026amp;amp;\u0026amp;amp; cp \u0026amp;#34;.openssl/lib/libcrypto.so\u0026amp;#34; /usr/lib/\n\n# Clone Nginx modules\nRUN mkdir /usr/src \\\n  \u0026amp;amp;\u0026amp;amp; cd /usr/src \\\n  \u0026amp;amp;\u0026amp;amp; git clone --depth=1 --recursive --shallow-submodules https://github.com/google/ngx_brotli \\\n  \u0026amp;amp;\u0026amp;amp; git clone --branch $NJS_VERSION --depth=1 --recursive --shallow-submodules https://github.com/nginx/njs\n\n# Build Nginx with modules\nRUN cd /usr/src \\\n  \u0026amp;amp;\u0026amp;amp; wget -qO nginx.tar.gz https://nginx.org/download/nginx-$NGINX_VERSION.tar.gz \\\n  \u0026amp;amp;\u0026amp;amp; tar -zxC /usr/src -f nginx.tar.gz \\\n  \u0026amp;amp;\u0026amp;amp; rm nginx.tar.gz \\\n  \u0026amp;amp;\u0026amp;amp; cd /usr/src/nginx-$NGINX_VERSION \\\n  \u0026amp;amp;\u0026amp;amp; mkdir /root/.cargo \\\n  \u0026amp;amp;\u0026amp;amp; echo $\u0026amp;#39;[net]\\ngit-fetch-with-cli = true\u0026amp;#39; \u0026amp;gt; /root/.cargo/config.toml \\\n  \u0026amp;amp;\u0026amp;amp; ./configure --prefix=/etc/nginx \\\n      --sbin-path=/usr/sbin/nginx \\\n      --modules-path=/usr/lib/nginx/modules \\\n      --conf-path=/etc/nginx/nginx.conf \\\n      --error-log-path=/var/log/nginx/error.log \\\n      --http-log-path=/var/log/nginx/access.log \\\n      --pid-path=/var/run/nginx.pid \\\n      --lock-path=/var/run/nginx.lock \\\n      --http-client-body-temp-path=/var/cache/nginx/client_temp \\\n      --http-proxy-temp-path=/var/cache/nginx/proxy_temp \\\n      --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp \\\n      --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp \\\n      --http-scgi-temp-path=/var/cache/nginx/scgi_temp \\\n      --user=nginx \\\n      --group=nginx \\\n      --with-pcre-jit \\\n      --with-http_ssl_module \\\n      --with-http_realip_module \\\n      --with-http_addition_module \\\n      --with-http_sub_module \\\n      --with-http_dav_module \\\n      --with-http_flv_module \\\n      --with-http_mp4_module \\\n      --with-http_gunzip_module \\\n      --with-http_gzip_static_module \\\n      --with-http_random_index_module \\\n      --with-http_secure_link_module \\\n      --with-http_stub_status_module \\\n      --with-http_auth_request_module \\\n      --with-http_xslt_module=dynamic \\\n      --with-http_image_filter_module=dynamic \\\n      --with-http_geoip_module=dynamic \\\n      --with-http_perl_module=dynamic \\\n      --with-threads \\\n      --with-stream \\\n      --with-stream_ssl_module \\\n      --with-stream_ssl_preread_module \\\n      --with-stream_realip_module \\\n      --with-stream_geoip_module=dynamic \\\n      --with-http_slice_module \\\n      --with-mail \\\n      --with-mail_ssl_module \\\n      --with-compat \\\n      --with-file-aio \\\n      --with-http_v2_module \\\n      --with-http_v3_module \\\n      --add-module=/usr/src/ngx_brotli \\\n      --add-module=/usr/src/njs/nginx \\\n      --with-cc-opt=\u0026amp;#34;-I$BORINGSSL/boringssl/include -Wno-error\u0026amp;#34; \\\n      --with-ld-opt=\u0026amp;#34;-L$BORINGSSL/boringssl/build/ssl -L$BORINGSSL/boringssl/build/crypto\u0026amp;#34; \\\n      --with-select_module \\\n      --with-poll_module \\\n      --build=\u0026amp;#34;docker-nginx-http3-$VCS_REF-$BUILD_DATE ngx_brotli-$(git --git-dir=/usr/src/ngx_brotli/.git rev-parse --short HEAD) njs-$(git --git-dir=/usr/src/njs/.git rev-parse --short HEAD)\u0026amp;#34; \\\n  \u0026amp;amp;\u0026amp;amp; touch \u0026amp;#34;$BORINGSSL/boringssl/.openssl/include/openssl/ssl.h\u0026amp;#34; \\\n  \u0026amp;amp;\u0026amp;amp; make -j$(getconf _NPROCESSORS_ONLN) \\\n  \u0026amp;amp;\u0026amp;amp; make install \\\n  \u0026amp;amp;\u0026amp;amp; rm -rf /etc/nginx/html/ \\\n  \u0026amp;amp;\u0026amp;amp; mkdir /etc/nginx/conf.d/ \\\n  \u0026amp;amp;\u0026amp;amp; mkdir -p /usr/share/nginx/html/ \\\n  \u0026amp;amp;\u0026amp;amp; install -m644 html/index.html /usr/share/nginx/html/ \\\n  \u0026amp;amp;\u0026amp;amp; install -m644 html/50x.html /usr/share/nginx/html/ \\\n  \u0026amp;amp;\u0026amp;amp; ln -s /usr/lib/nginx/modules /etc/nginx/modules \\\n  \u0026amp;amp;\u0026amp;amp; strip /usr/sbin/nginx* \\\n  \u0026amp;amp;\u0026amp;amp; strip /usr/lib/nginx/modules/*.so \\\n  \u0026amp;amp;\u0026amp;amp; rm -rf /etc/nginx/*.default /etc/nginx/*.so \\\n  \u0026amp;amp;\u0026amp;amp; rm -rf /usr/src \\\n  \u0026amp;amp;\u0026amp;amp; apk add --no-cache --virtual .gettext gettext \\\n  \u0026amp;amp;\u0026amp;amp; mv /usr/bin/envsubst /tmp/ \\\n  \\\n  # Determine runtime dependencies\n  \u0026amp;amp;\u0026amp;amp; runDeps=\u0026amp;#34;$( \\\n      scanelf --needed --nobanner /usr/sbin/nginx /usr/lib/nginx/modules/*.so /tmp/envsubst \\\n      | awk \u0026amp;#39;{ gsub(/,/, \u0026amp;#34;\\nso:\u0026amp;#34;, $2); print \u0026amp;#34;so:\u0026amp;#34; $2 }\u0026amp;#39; \\\n      | sort -u \\\n      | xargs -r apk info --installed \\\n      | sort -u \\\n    )\u0026amp;#34; \\\n  \u0026amp;amp;\u0026amp;amp; echo \u0026amp;#34;$runDeps\u0026amp;#34; \u0026amp;gt; /nginx-run-deps.txt \\\n  \\\n  # Cleanup\n  \u0026amp;amp;\u0026amp;amp; apk del .brotli-build-deps .build-deps .gettext \\\n  \u0026amp;amp;\u0026amp;amp; rm -rf /root/.cargo \\\n  \u0026amp;amp;\u0026amp;amp; rm -rf /var/cache/apk/* \\\n  \u0026amp;amp;\u0026amp;amp; mv /tmp/envsubst /usr/local/bin/ \\\n  \u0026amp;amp;\u0026amp;amp; mkdir -p /etc/ssl/private \\\n  \u0026amp;amp;\u0026amp;amp; openssl req -x509 -newkey rsa:4096 -nodes -keyout /etc/ssl/private/localhost.key -out /etc/ssl/localhost.pem -days 365 -sha256 -subj \u0026amp;#39;/CN=localhost\u0026amp;#39; \\\n  \u0026amp;amp;\u0026amp;amp; ln -sf /dev/stdout /var/log/nginx/access.log \\\n  \u0026amp;amp;\u0026amp;amp; ln -sf /dev/stderr /var/log/nginx/error.log\n\n# Final Stage\nFROM alpine:latest\n\n# Copy Nginx and related files from the builder stage\nCOPY --from=builder /usr/sbin/nginx /usr/sbin/nginx\nCOPY --from=builder /etc/nginx /etc/nginx\nCOPY --from=builder /usr/lib/nginx /usr/lib/nginx\nCOPY --from=builder /usr/share/nginx /usr/share/nginx\nCOPY --from=builder /usr/local/bin/envsubst /usr/local/bin/envsubst\nCOPY --from=builder /var/log/nginx /var/log/nginx\nCOPY --from=builder /var/cache/nginx /var/cache/nginx\nCOPY --from=builder /var/run /var/run\nCOPY --from=builder /etc/ssl /etc/ssl\nCOPY --from=builder /usr/lib/libssl.so* /usr/lib/\nCOPY --from=builder /usr/lib/libcrypto.so* /usr/lib/\nCOPY --from=builder /nginx-run-deps.txt /nginx-run-deps.txt\n\n# Install runtime dependencies\nRUN apk add --no-cache ca-certificates openssl libstdc\u0026#43;\u0026#43; libgcc \\\n  \u0026amp;amp;\u0026amp;amp; apk add --no-cache $(cat /nginx-run-deps.txt) \\\n  \u0026amp;amp;\u0026amp;amp; rm /nginx-run-deps.txt \\\n  \u0026amp;amp;\u0026amp;amp; update-ca-certificates \\\n  \\\n  # Forward logs to Docker\n  \u0026amp;amp;\u0026amp;amp; ln -sf /dev/stdout /var/log/nginx/access.log \\\n  \u0026amp;amp;\u0026amp;amp; ln -sf /dev/stderr /var/log/nginx/error.log\n\n# Create nginx user and group\nRUN addgroup -S nginx \\\n  \u0026amp;amp;\u0026amp;amp; adduser -D -S -h /var/cache/nginx -s /sbin/nologin -G nginx nginx\n\n# Expose ports\nEXPOSE 80 443\n\nSTOPSIGNAL SIGTERM\n\nCMD [\u0026amp;#34;nginx\u0026amp;#34;, \u0026amp;#34;-g\u0026amp;#34;, \u0026amp;#34;daemon off;\u0026amp;#34;]\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch2 id=\"nginxconf-sample\"\u003eNginx.conf sample\u003c/h2\u003e\n\u003cp\u003eHere you will find a sample nginx.conf which give you an idea of how i run my nginx environment.\u003c/p\u003e\n\u003cp\u003eThe most important difference in this file compared to the normal nginx.conf should be the brotli compression settings.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-nginx\"\u003e\u003ccode class=\"language-nginx\"\u003euser nginx;\nworker_processes auto;\n\nerror_log  /var/log/nginx/error.log notice;\npid        /var/run/nginx.pid;\n\ninclude /etc/nginx/modules-enabled/*.conf;\n\nevents {\n\tworker_connections 1024;\n}\n\nhttp {\n\t##\n\t# Basic Settings\n\t##\n\tsendfile on;\n\ttcp_nopush on;\n\ttcp_nodelay on;\n\tkeepalive_timeout 65;\n\ttypes_hash_max_size 2048;\n\n\tproxy_connect_timeout  300s;\n\tproxy_send_timeout  300s;\n\tproxy_read_timeout  300s;\n\tfastcgi_send_timeout 300s;\n\tfastcgi_read_timeout 300s;\n\n\tinclude /etc/nginx/mime.types;\n\tdefault_type application/octet-stream;\n\n\t##\n\t# SSL Settings\n\t##\n\tssl_protocols TLSv1.2 TLSv1.3;\n\tssl_prefer_server_ciphers on;\n\n\t##\n\t# Logging Settings\n\t##\n    log_format  main  \u0026amp;#39;$remote_addr - $remote_user [$time_local] \u0026amp;#34;$request\u0026amp;#34; \u0026amp;#39;\n                      \u0026amp;#39;$status $body_bytes_sent \u0026amp;#34;$http_referer\u0026amp;#34; \u0026amp;#39;\n                      \u0026amp;#39;\u0026amp;#34;$http_user_agent\u0026amp;#34; \u0026amp;#34;$http_x_forwarded_for\u0026amp;#34;\u0026amp;#39;;\n\n    access_log  /var/log/nginx/access.log  main;\n\n\t##\n\t# Gzip Settings\n\t##\n\tgzip on;\n\tgzip_vary on;\n\tgzip_proxied any;\n\tgzip_comp_level 6;\n\tgzip_buffers 16 8k;\n\tgzip_http_version 1.1;\n\tgzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml\u0026#43;rss text/javascript;\n\n\t##\n\t# Brotli Settings (modify the brotli_comp_level to match your optimal cpu load vs compression)\n\t##\n\tbrotli on;\n\tbrotli_static on;\n\tbrotli_comp_level 11;\n\tbrotli_types\n\ttext/plain\n\ttext/css\n\ttext/xml\n\ttext/javascript\n\ttext/x-component\n\tapplication/xml\n\tapplication/xml\u0026#43;rss\n\tapplication/javascript\n\tapplication/json\n\tapplication/atom\u0026#43;xml\n\tapplication/vnd.ms-fontobject\n\tapplication/x-font-ttf\n\tapplication/x-font-opentype\n\tapplication/x-font-truetype\n\tapplication/x-web-app-manifest\u0026#43;json\n\tapplication/xhtml\u0026#43;xml\n\tapplication/octet-stream\n\tfont/opentype\n\tfont/truetype\n\tfont/eot\n\tfont/otf\n\tfont/woff\n\tfont/woff2\n\timage/svg\u0026#43;xml\n\timage/x-icon\n\timage/vnd.microsoft.icon\n\timage/bmp;\n\n\t##\n\t# Virtual Host Configs\n\t##\n\tinclude /etc/nginx/conf.d/*.conf;\n}\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch2 id=\"siteconf-sample\"\u003eSite.conf sample\u003c/h2\u003e\n\u003cp\u003eYou need to watch out that the higher you set the brotli compression the more cpu intensive it becomes. TTFB (time to first byte) can become an issue when the compression ratio is set to high with to slow kubernetes/ high loads or everything in between. so you need to test on real world scenario\u0026rsquo;s and adjust accordingly. This is not only a Brotli issue but Gzip also has the same disadvantages when setting compression levels to high while the decrease in file size isn\u0026rsquo;t worth it.\u003c/p\u003e\n\u003cp\u003eYou can use this file to take all the HTTP/3 settings defined here below. This will optimize and enable HTTP/3 support for your site.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-nginx\"\u003e\u003ccode class=\"language-nginx\"\u003eserver {\n       \t# HTTP/3 and Quic Listen\n      \tlisten 443 quic reuseport;\n        http3 on;\n\n        # HTTP/3 Add Alt-Svc header negotiation.\n        add_header alt-svc \u0026amp;#39;h2=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3-29=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3=\u0026amp;#34;:443\u0026amp;#34;; ma=86400\u0026amp;#39; always;\n        \n        # HTTP/2 Listen\n        listen 443 ssl http2;\n        http2 on;\n\n        # Certificate locations for HTTPS.\n\t    ssl_certificate /etc/ssl/certs/wildcard-cert-23.crt;\n        ssl_certificate_key /etc/ssl/private/wildcard-cert-23.key;\n\n        # Enable SSL settings for performance.\n        ssl_session_cache shared:SSL:50m;\n        ssl_session_timeout 1d;\n        ssl_session_tickets off;\n        # Enable SSL settings for security.\n        ssl_prefer_server_ciphers on;\n        ssl_protocols TLSv1.3;\n        ssl_ciphers HIGH:!aNULL:!MD5;\n\n        # Base settings (root and hostname).\n        root /var/www/html;\n        index index.php index.html index.htm;\n        server_name kubernetes.servername.nl;\n\n        # HTTP/2 settings\n        http2_push_preload on;\n        # /End of HTTP/2.\n\n        # HTTP/3 settings\n        # Enable TLSv1.3\u0026amp;#39;s 0-RTT. Use $ssl_early_data when reverse proxying to\n        # prevent replay attacks.\n        # @see: http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_early_data\n        ssl_early_data on;\n        quic_retry on;\n        quic_gso on;\n        quic_host_key /etc/ssl/private/wildcard-cert-23.key;\n        # /End of HTTP/3.\n\n        location / {\n                try_files $uri $uri/ /index.php?$query_string;\n\t    }\n\n        location ~ \\.php$ {\n                try_files $uri =404;\n                # Bunch of PHP and FPM settings\n                fastcgi_split_path_info ^(.\u0026#43;\\.php)(/.\u0026#43;)$;\n                fastcgi_pass your-php-fpm-instance:9000;\n                fastcgi_index index.php;\n                include fastcgi_params;\n                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;\n                fastcgi_param PATH_INFO $fastcgi_path_info\n                \n                # Add a bunch of other headers based on your desired settings.\n                add_header X-protocol $server_protocol always;   \n                add_header X-Cache $upstream_cache_status always;\n                add_header X-Frame-Options SAMEORIGIN always;\n                add_header X-Content-Type-Options nosniff always;\n                add_header X-XSS-Protection \u0026amp;#34;1; mode=block\u0026amp;#34;;\n                add_header Strict-Transport-Security \u0026amp;#34;max-age=31536000; includeSubdomains; preload\u0026amp;#34; always;\n                add_header Referrer-Policy strict-origin-when-cross-origin;\n                add_header Pragma public;\n                add_header Cache-Control \u0026amp;#34;public\u0026amp;#34;;\n        }\n}\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch2 id=\"multiple-h3-connections-from-a-single-nginx\"\u003eMultiple H3 Connections from a single Nginx\u003c/h2\u003e\n\u003cp\u003eHere i will explain how you can make multiple server blocks within Nginx work with HTTP/3. This has some caveats since http3 uses UDP with the setting reuseport which makes it unable to assign port 443 multiple times when using HTTP/3\u003c/p\u003e\n\u003cpre class=\"line-numbers language-nginx\"\u003e\u003ccode class=\"language-nginx\"\u003eadd_header alt-svc \u0026amp;#39;h2=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3-29=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3=\u0026amp;#34;:443\u0026amp;#34;; ma=86400\u0026amp;#39; always;\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eThis part of the header informs that our HTTP/3 connection accepts port 443. When you have a single nginx container setup and you have more server blocks configured you will get issues regarding the reuse for port 443.\u003c/p\u003e\n\u003cp\u003eWith Http3 you can only define a single port once within a server block. So if you have more server blocks you have the option to let your website communicate over different ports then 443 using this alt-svc header which tells the browser it accepts HTTP/3 and on which port.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-nginx\"\u003e\u003ccode class=\"language-nginx\"\u003eadd_header alt-svc \u0026amp;#39;h2=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3-29=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3=\u0026amp;#34;:443\u0026amp;#34;; ma=86400\u0026amp;#39; always;\n\nadd_header alt-svc \u0026amp;#39;h2=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3-29=\u0026amp;#34;:444\u0026amp;#34;; ma=86400, h3=\u0026amp;#34;:444\u0026amp;#34;; ma=86400\u0026amp;#39; always;\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eCheck out the differences in the h3 ports here. This way you can make multiple server blocks use h3 by specifying unique ports.\u003c/p\u003e\n\u003cp\u003eAll you need to do is make your instance listen to the right port specified in your alt-svc header. If there is a successful connection possible the browser will try to switch to h3. Below a sample how you can listen to port 443 and 444 with HTTP3\u003c/p\u003e\n\u003cpre class=\"line-numbers language-nginx\"\u003e\u003ccode class=\"language-nginx\"\u003eserver {\n\n        server_name http3-port-443\n\n       \t# HTTP/3 and Quic Listen\n      \tlisten 443 quic reuseport;\n        http3 on;\n\n        # HTTP/3 Add Alt-Svc header negotiation.\n        add_header alt-svc \u0026amp;#39;h2=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3-29=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3=\u0026amp;#34;:443\u0026amp;#34;; ma=86400\u0026amp;#39; always;\n        \n        # HTTP/2 Listen\n        listen 443 ssl http2;\n        http2 on;\n        \n        # HTTP/3 settings\n        # Enable TLSv1.3\u0026amp;#39;s 0-RTT. Use $ssl_early_data when reverse proxying to\n        # prevent replay attacks.\n        # @see: http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_early_data\n        ssl_early_data on;\n        quic_retry on;\n        quic_gso on;\n        quic_host_key /etc/ssl/private/wildcard-cert-23.key;\n        # /End of HTTP/3.\n\n        location / {\n                try_files $uri $uri/ /index.php?$query_string;\n\t}\n}\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cpre class=\"line-numbers language-nginx\"\u003e\u003ccode class=\"language-nginx\"\u003eserver {\n\n        server_name http3-port-444\n\n       \t# HTTP/3 and Quic Listen\n      \tlisten 444 quic reuseport;\n        http3 on;\n\n        # HTTP/3 Add Alt-Svc header negotiation.\n        add_header alt-svc \u0026amp;#39;h2=\u0026amp;#34;:443\u0026amp;#34;; ma=86400, h3-29=\u0026amp;#34;:444\u0026amp;#34;; ma=86400, h3=\u0026amp;#34;:444\u0026amp;#34;; ma=86400\u0026amp;#39; always;\n        \n        # HTTP/2 Listen\n        listen 443 ssl http2;\n        http2 on;\n        \n        # HTTP/3 settings\n        # Enable TLSv1.3\u0026amp;#39;s 0-RTT. Use $ssl_early_data when reverse proxying to\n        # prevent replay attacks.\n        # @see: http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_early_data\n        ssl_early_data on;\n        quic_retry on;\n        quic_gso on;\n        quic_host_key /etc/ssl/private/wildcard-cert-23.key;\n        # /End of HTTP/3.\n\n        location / {\n                try_files $uri $uri/ /index.php?$query_string;\n\t}\n}\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eIt is that easy to make your Reverse proxy or multiple server blocks establish a H3 connection. You do have to make the right port forwards within your firewall/network also.\u003c/p\u003e\n\u003cp\u003eWhen using docker you configure this as shown below. When specifying UDP you have to specify every port seperately in docker. Below is my sample which includes UDP ports 443 up to 449.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-yaml\"\u003e\u003ccode class=\"language-yaml\"\u003eservices:\n  ############################################################\n  # proxy webservice to connect to multiple nginx instances\n  ############################################################\n  reverse-proxy:\n    image: custom-build-nginx:latest\n    restart: unless-stopped\n    ports:\n      - \u0026amp;#39;80:80\u0026amp;#39;\n      - \u0026amp;#39;443:443\u0026amp;#39;\n      - \u0026amp;#39;443:443/udp\u0026amp;#39;\n      - \u0026amp;#39;444:444/udp\u0026amp;#39;\n      - \u0026amp;#39;445:445/udp\u0026amp;#39;\n      - \u0026amp;#39;446:446/udp\u0026amp;#39;\n      - \u0026amp;#39;447:447/udp\u0026amp;#39;\n      - \u0026amp;#39;448:448/udp\u0026amp;#39;\n      - \u0026amp;#39;449:449/udp\u0026amp;#39;\n    volumes:\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch2 id=\"update-03-04-2024\"\u003eUpdate (03-04-2024)\u003c/h2\u003e\n\u003cp\u003eAdded some versioning to my dockerfile to better compare versions. I also cleaned up the build from yesterday and finally decided to add a ENV for setting the boringssl version also. Since their build broke my dockerfile we better update it manually. I also updated nginx to 1.25.4 and NJS to 0.8.3 and verified the build\u003c/p\u003e\n\u003ch2 id=\"update-02-04-2024\"\u003eUpdate (02-04-2024)\u003c/h2\u003e\n\u003cp\u003eI only recently found out that an update to BoringSSL done around 28 february broke my build of BoringSSL. below the updated dockerfile that compiles again. I had to build libssl as a shared library for it to function again. I will cleanup this image a bit more but it is at least working again.\u003c/p\u003e\n\u003ch2 id=\"update-18-03-2024\"\u003eUpdate (18-03-2024)\u003c/h2\u003e\n\u003cp\u003eAdded a section to explain how you can use multiple H3 connections when you have a Nginx that has more than 1 server block. This also holds the key when you use Nginx as a Reverse Proxy and want to make use of H3.\u003c/p\u003e\n\u003ch2 id=\"update-04-09-2023\"\u003eUpdate (04-09-2023)\u003c/h2\u003e\n\u003cp\u003eI found out that this build didnt compile anymore due to a change in the brotli source code? It might had something to do with updates from github i didnt version in this Dockerfile. I completely reworked this file to work with the updated Nginx 1.25.2 which has \u0026ldquo;built in\u0026rdquo; HTTP/3 support. I also made sure we have native Brotli support again. This should keep functioning from now on.\u003c/p\u003e\n\u003ch2 id=\"update-30-08-2023\"\u003eUpdate (30-08-2023)\u003c/h2\u003e\n\u003cp\u003eI removed the modsec support there was before. This was giving random Errors and since i wont use it i removed it. It might return later.\u003c/p\u003e\n","date_modified":"2024-11-13T12:22:12+01:00","date_published":"2023-03-20T11:20:00+01:00","id":"https://www.allsubjectsmatter.nl/docker-nginx-http3-and-brotli/","language":"en","summary":"Here you will find a working docker file i created as a base to build my projects on. This a recent Nginx with compiled HTTP/3 support and Brotli support for better compression. This is posted here to share it back with anyone.","tags":["Under the hood"],"title":"Docker Nginx HTTP/3 and Brotli","url":"https://www.allsubjectsmatter.nl/docker-nginx-http3-and-brotli/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eI struggled for a long time with the idea of starting a blog. Not just to have a blog but to be able to share my perspective on certain subjects with anyone who finds it useful. This is my first attempt to explain my way of thinking and working.\u003c/p\u003e\n\u003cp\u003eThe idea to explain how we at 333travel are achieving FLOW triggered me while reading the book \u003ca href=\"https://www.goodreads.com/book/show/59691869-applying-scrum-with-kanban\" data-external\u003eA\u003cem\u003epplying Scrum with kanban\u003c/em\u003e\u003c/a\u003e. As I started reading, I realized that many of the ideas presented in this book were ones that we were already using. Reading about these ideas gave me the courage to write this from my own perspective.\u003c/p\u003e\n\u003cp\u003eI am writing this post halfway in the book. Where Katie is exploring positive FLOW with her team where she acts as a Scrum Master. When the book started talking about FLOW, something deep inside of me got triggered.\u003c/p\u003e\n\u003cp\u003eI got the chills reading this chapter because I just realized that we already implemented a lot of the ideas in the book. Not knowing what Scrum or Kanban was in my early days. Just by trying what felt like a natural and good way to work, back in 2017 using agile methods.\u003c/p\u003e\n\u003ch2 id=\"outcomes\"\u003eOutcomes\u003c/h2\u003e\n\u003cp\u003eWe have made a lot of progress since then. I can already say we are practicing a lot of the outcomes that the book uncovers.\u003c/p\u003e\n\u003cp\u003eWe started working with Jira in 2017 using a modified version of swim lanes.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eHaving a board with swimming lanes that cover the topics of issues.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eScrapping some swimming lanes since maintenance needs to be controllable and there has to be a use for them.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eDuring the years that followed and when we really wanted to professionalize our way of working the following subjects entered our workflow. These insights are in no particular order.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWriting a definition of done (DOD) for each of the swimming lane states.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eHaving the sprint goal represented on our physical whiteboard.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eUsing a physical whiteboard that is the single source of truth over Jira.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eHaving specific colors on the stories we pick up within a sprint, with a special color (green) for the stories that contribute the most to the sprint goal.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eUsing specific colors for other tasks, hotfixes, and such.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eBlocked issues aren\u0026rsquo;t a state where they can reside. Blocked issues need to get unblocked as soon as possible or rolled back after confirming that we can\u0026rsquo;t work further.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cblockquote\u003e\n\u003cp\u003eWorking from a feeling that we need to finish off every task/story that is swimming through the lanes as a collective to the far right before we decide to start new work (stories).\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eFor an visual insight on our current Scrum board i refer you to the bookmark at the end of this post.\u003c/p\u003e\n\u003ch2 id=\"get-a-flow-going\"\u003eGet a FLOW going!\u003c/h2\u003e\n\u003cp\u003eWhy i got triggered kept me busy for a long time. Thinking about it now, I understand that they were the catalysts to get our flow going. What I saw in my team was that only when people started really working together. (As in working on the same stories and picking up their subtasks within it.) Only then we really opened the way for conversations about the stories we are working on.\u003c/p\u003e\n\u003cp\u003eBut in order to be able to do that we had to create a certain way of working first, covering all of the items listed above to foster transparency and focus.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eScrum is a talk-facilitating framework. To get a flow going you need to collaborate. Working on the same stories with all developers towards the sprint goal. Ideally, you should finish something before moving on to something new.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eDevelopers often try to avoid really working together, thinking it creates more work. They often prefer to split stories further so they can work solo on stories. Instead of working on subtasks within the same story. This small change in behavior takes some getting used to for developers. But when they start operating as a team, they can enter a state of positive flow in which everything works.\u003c/p\u003e\n\u003cp\u003eIf you keep doing this solo Scrum within a sprint, you\u0026rsquo;ll lose many of the benefits Scrum can provide. You can of course say you are doing Scrum. But do you want to say you\u0026rsquo;re doing Scrum? Or do you want to get in an epic flow together and forever change the way you work! (who cares what the name is, right)?\u003c/p\u003e\n\u003cp\u003eIt\u0026rsquo;s our standard behavior to start working on tasks we already know best within a sprint (it\u0026rsquo;s quicker to finish that way, right?). I also challenge my developers to work on things they haven\u0026rsquo;t worked on yet within our projects. We need to get better as a Scrum team. This won\u0026rsquo;t happen if we keep working in our own knowledge bubbles. Developers need to be challenged to work together towards the product goal. This will give each developer more autonomy and confidence.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eScrum isn\u0026rsquo;t about the speed of development. this is merely a benefit when you reach a certain state of flow with your team.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003ch2 id=\"instill-a-vision\"\u003eInstill a vision\u003c/h2\u003e\n\u003cp\u003eInstilling a vision is also crucial for FLOW. Having a idea of why you are building this product is one of the most important things to start with. A product vision or goal and a clear sprint goal to start from are a must. You need to be able to communicate your goal. This way you know what we are working on and what we need to do next to achieve that goal.\u003c/p\u003e\n\u003cblockquote class=\"pull-quote\"\u003e\u003cp\u003eDefining technical debt is heavily relying on your product vision/goal. This enables you to compare your current product state to where you want it to be. And what technical debt we need to solve to get there.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eA product vision isn\u0026rsquo;t concrete, and it doesn\u0026rsquo;t have to be long. It will evolve over time as you discover more about what the product will do and what the market wants. But it all starts with a vision you\u0026rsquo;re working towards. There needs to be something to explain WHY.\u003c/p\u003e\n\u003ch2 id=\"conclusion\"\u003eConclusion\u003c/h2\u003e\n\u003cp\u003eI hope i shared some useful insights in how we were able to achieve FLOW in our Scrum team. We are merely scratching the surface in my opinion, and a lot more optimizing still has to be addressed in our team. But we are are for sure on the right track.\u003c/p\u003e\n\u003cp\u003eEspecially since our whole Scrum team has the feeling that we are operating in a FLOW. This is a pretty darn good feeling to work in!\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.allsubjectsmatter.nl/sprints-in-real-life/\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eOur sprints in a real life example\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eThis post explains how we as a team manage our current Board on which we keep our sprints. This board is leading over Jira we love working analog next to our digital work.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/achieving-flow-with-scrum/bookmarks/01-icon_hu_9b466dd874965fcb.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eAll Subjects Matter\u003c/span\u003e\u003cspan\u003eRemco\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/achieving-flow-with-scrum/bookmarks/01-thumb_hu_12ce7537b849d91a.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n","date_modified":"2024-02-16T08:45:37+01:00","date_published":"2023-03-07T10:20:56+01:00","id":"https://www.allsubjectsmatter.nl/achieving-flow-with-scrum/","language":"en","summary":"In this post I share our current journey of implementing Scrum and Kanban principles with my team, this is mainly about emphasizing the importance of real collaborative work and instilling a product vision which helps us to achieve FLOW which translates to high value Scrum for us.","tags":["Teams"],"title":"Achieving FLOW with Scrum","url":"https://www.allsubjectsmatter.nl/achieving-flow-with-scrum/"},{"authors":[{"name":"Remco Loup","url":"https://www.allsubjectsmatter.nl/author/remco/"}],"content_html":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eIn this post i will explain my particular setup of how i use my windows terminal along with some other tips. So if you want to impress someone while working in your favorite coffee place by making them feel like you work for Nasa. You are in the right place! All jokes aside, working with something day in day out it at least may look more appealing if possible, right?\u003c/p\u003e\n\u003cp\u003eLets start with a list below of everything i have installed after which i will explain what they all do.\u003c/p\u003e\n\u003ch2 id=\"zsh-and-the-p10k-theme\"\u003eZSH and the p10k theme\u003c/h2\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://github.com/ohmyzsh/ohmyzsh\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eGitHub - ohmyzsh/ohmyzsh: 🙃 A delightful community-driven (with 2,100\u0026#43; contributors) framework for managing your zsh configuration. Includes 300\u0026#43; optional plugins (rails, git, macOS, hub, docker, homebrew, node, php, python, etc), 140\u0026#43; themes to spice up your morning, and an auto-update tool so that makes it easy to keep up with the latest updates from the community.\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003e🙃 A delightful community-driven (with 2,100\u0026#43; contributors) framework for managing your zsh configuration. Includes 300\u0026#43; optional plugins (rails, git, macOS, hub, docker, homebrew, node, php, pyth…\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/01-icon_hu_a8d8b8bb716d7df8.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eGitHub\u003c/span\u003e\u003cspan\u003eohmyzsh\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/01-thumb_hu_cd46afdcd6f277f0.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://github.com/romkatv/powerlevel10k\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eGitHub - romkatv/powerlevel10k: A Zsh theme\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eA Zsh theme. Contribute to romkatv/powerlevel10k development by creating an account on GitHub.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/02-icon_hu_a8d8b8bb716d7df8.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eGitHub\u003c/span\u003e\u003cspan\u003eromkatv\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/02-thumb_hu_e0ce4be87804921b.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://spacevim.org/quick-start-guide/\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eQuick start guide | SpaceVim\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eA quick start guide which will tell you how to install and configure SpaceVim, also provides a list of resources for learning SpaceVim.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cspan\u003eSpaceVim\u003c/span\u003e\u003cspan\u003eShidong Wang\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/03-thumb_hu_fd18ea080be99d12.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cp\u003eI have installed the above items for the look and feel of my Shell. I use Windows Terminal and this makes my terminal look good and distinguishes the right information with colors added to the shell.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eohmyzsh\u003c/strong\u003eis the framework for managing my ZSH configuration. The theme i installed is p10k. This is a highly customizable theme.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSpaceVim\u003c/strong\u003eis one the of the best visual editors for editing everything you need in Linux. Go check it out by using the link.\u003c/p\u003e\n\u003cp\u003eHere a screenshot of how my personal shell looks like. I will also share the Json config below of my windows terminal. The font i use is the default nerd font which comes with p10k. This is the Meslo Gorgeous monospace Nerd font.\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/images/image-1_hu_98a6094c3d3897a5.webp\" alt=\"\" width=\"560\" height=\"342\" data-zoom-src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/images/image-1_hu_f6fef50f1a2399a5.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003cp\u003eThe Colors i use are called Atom One Dark. And are a subtle pastel colored font\u003c/p\u003e\n\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/images/image_hu_6020c119054673fb.webp\" alt=\"\" width=\"400\" height=\"70\" data-zoom-src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/images/image_hu_6020c119054673fb.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/figure\u003e\n\n\u003cp\u003eIn the json file below you can find some nice benefits in making your experience even more customized.\u003c/p\u003e\n\u003ch2 id=\"kali-linux\"\u003eKali linux\u003c/h2\u003e\n\u003cp\u003eI have Kali linux installed working in a seamless overlay. This means you can run Kali visually native within your Windows. See the screenshot below how neat this looks! Follow the link below for how to get this working with Kali.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://www.kali.org/docs/wsl/win-kex/\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eWin-KeX | Kali Linux Documentation\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eWindows Subsystem for Linux 2 \u0026amp; Win-KeX\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/04-icon_hu_5cd2a9b225aaa8ff.webp\" alt=\"\" class=\"bookmark__icon\" width=\"32\" height=\"32\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eKali Linux\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/04-thumb.svg\" sizes=\"250px\" alt=\"\" loading=\"lazy\" decoding=\"async\"\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cpre class=\"line-numbers language-json\"\u003e\u003ccode class=\"language-json\"\u003e\u0026amp;#34;commandline\u0026amp;#34;: \u0026amp;#34;wsl -d kali-linux kex --sl --wtstart -s\u0026amp;#34;,\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cfigure class=\"figure\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003csource type=\"image/webp\" sizes=\"(max-width: 47.99rem) calc(100vw - 2.5rem), 720px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/images/image-2_hu_727fbce17435cc82.webp\" alt=\"\" width=\"560\" height=\"351\" data-zoom-src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/images/image-2_hu_b5bbf743fe36003f.webp\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cfigcaption\u003eSeamless integration with kali and Windows\u003c/figcaption\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"external-ssh-in-windows-terminal\"\u003eExternal SSH in Windows Terminal\u003c/h2\u003e\n\u003cp\u003eYou can also easily login to (external) servers requiring tokens with Windows Terminal.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-json\"\u003e\u003ccode class=\"language-json\"\u003e\u0026amp;#34;commandline\u0026amp;#34;: \u0026amp;#34;ssh -i \\\u0026amp;#34;~/.ssh/id_rsa\\\u0026amp;#34; username@ip-address\u0026amp;#34;,\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003cp\u003eLook at how to configure this using the link below.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://blog.anurut.com/ssh-with-private-key-in-windows-terminal/\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eHow to SSH into a remote server using private key and Windows Terminal\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eSSH into a remote computer in just one click using a private key and Windows Terminal | Super easy.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cspan\u003e./anurut\u0026#39;s-blog\u003c/span\u003e\u003cspan\u003eAnurut\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003ch2 id=\"windows-terminal-config\"\u003eWindows Terminal config\u003c/h2\u003e\n\u003cp\u003eHere is the json for the Windows Terminal with the last part the color theme i am using. and the settings for the look and feel i use. Also present are examples for running kali seamless and an SSH session with tokens.\u003c/p\u003e\n\u003cpre class=\"line-numbers language-json\"\u003e\u003ccode class=\"language-json\"\u003e    \u0026amp;#34;profiles\u0026amp;#34;: \n    {\n        \u0026amp;#34;defaults\u0026amp;#34;: \n        {\n            \u0026amp;#34;antialiasingMode\u0026amp;#34;: \u0026amp;#34;cleartype\u0026amp;#34;,\n            \u0026amp;#34;backgroundImage\u0026amp;#34;: null,\n            \u0026amp;#34;backgroundImageAlignment\u0026amp;#34;: \u0026amp;#34;bottom\u0026amp;#34;,\n            \u0026amp;#34;backgroundImageOpacity\u0026amp;#34;: 0.8,\n            \u0026amp;#34;colorScheme\u0026amp;#34;: \u0026amp;#34;Atom One Dark\u0026amp;#34;,\n            \u0026amp;#34;cursorShape\u0026amp;#34;: \u0026amp;#34;vintage\u0026amp;#34;,\n            \u0026amp;#34;experimental.retroTerminalEffect\u0026amp;#34;: false,\n            \u0026amp;#34;font\u0026amp;#34;: \n            {\n                \u0026amp;#34;face\u0026amp;#34;: \u0026amp;#34;MesloLGS NF\u0026amp;#34;\n            },\n            \u0026amp;#34;opacity\u0026amp;#34;: 75,\n            \u0026amp;#34;useAcrylic\u0026amp;#34;: true\n        },\n        \u0026amp;#34;list\u0026amp;#34;: \n        [\n            {\n                \u0026amp;#34;commandline\u0026amp;#34;: \u0026amp;#34;wsl -d kali-linux kex --sl --wtstart -s\u0026amp;#34;,\n                \u0026amp;#34;guid\u0026amp;#34;: \u0026amp;#34;{55ca431a-3a87-5fb3-83cd-11ececc031d2}\u0026amp;#34;,\n                \u0026amp;#34;hidden\u0026amp;#34;: false,\n                \u0026amp;#34;icon\u0026amp;#34;: \u0026amp;#34;C:\\\\Users\\\\icons\\\\kali-menu.png\u0026amp;#34;,\n                \u0026amp;#34;name\u0026amp;#34;: \u0026amp;#34;Kali Seamless overlay\u0026amp;#34;,\n                \u0026amp;#34;startingDirectory\u0026amp;#34;: \u0026amp;#34;%USERPROFILE%\u0026amp;#34;\n            },\n            {\n                \u0026amp;#34;commandline\u0026amp;#34;: \u0026amp;#34;ssh -i \\\u0026amp;#34;~/.ssh/id_rsa\\\u0026amp;#34; username@ip-address\u0026amp;#34;,\n                \u0026amp;#34;guid\u0026amp;#34;: \u0026amp;#34;{0caa0dad-35be-5f56-a8ff-afceaeaa6103}\u0026amp;#34;,\n                \u0026amp;#34;hidden\u0026amp;#34;: false,\n                \u0026amp;#34;name\u0026amp;#34;: \u0026amp;#34;External test server\u0026amp;#34;\n            }\n        ]\n    },\n    \u0026amp;#34;schemes\u0026amp;#34;: \n    [\n        {\n            \u0026amp;#34;background\u0026amp;#34;: \u0026amp;#34;#282C34\u0026amp;#34;,\n            \u0026amp;#34;black\u0026amp;#34;: \u0026amp;#34;#000000\u0026amp;#34;,\n            \u0026amp;#34;blue\u0026amp;#34;: \u0026amp;#34;#61AFEF\u0026amp;#34;,\n            \u0026amp;#34;brightBlack\u0026amp;#34;: \u0026amp;#34;#5C6370\u0026amp;#34;,\n            \u0026amp;#34;brightBlue\u0026amp;#34;: \u0026amp;#34;#61AFEF\u0026amp;#34;,\n            \u0026amp;#34;brightCyan\u0026amp;#34;: \u0026amp;#34;#56B6C2\u0026amp;#34;,\n            \u0026amp;#34;brightGreen\u0026amp;#34;: \u0026amp;#34;#98C379\u0026amp;#34;,\n            \u0026amp;#34;brightPurple\u0026amp;#34;: \u0026amp;#34;#C678DD\u0026amp;#34;,\n            \u0026amp;#34;brightRed\u0026amp;#34;: \u0026amp;#34;#E06C75\u0026amp;#34;,\n            \u0026amp;#34;brightWhite\u0026amp;#34;: \u0026amp;#34;#FFFFFF\u0026amp;#34;,\n            \u0026amp;#34;brightYellow\u0026amp;#34;: \u0026amp;#34;#D19A66\u0026amp;#34;,\n            \u0026amp;#34;cursorColor\u0026amp;#34;: \u0026amp;#34;#FFFFFF\u0026amp;#34;,\n            \u0026amp;#34;cyan\u0026amp;#34;: \u0026amp;#34;#56B6C2\u0026amp;#34;,\n            \u0026amp;#34;foreground\u0026amp;#34;: \u0026amp;#34;#CCCCCC\u0026amp;#34;,\n            \u0026amp;#34;green\u0026amp;#34;: \u0026amp;#34;#98C379\u0026amp;#34;,\n            \u0026amp;#34;name\u0026amp;#34;: \u0026amp;#34;Atom One Dark\u0026amp;#34;,\n            \u0026amp;#34;purple\u0026amp;#34;: \u0026amp;#34;#C678DD\u0026amp;#34;,\n            \u0026amp;#34;red\u0026amp;#34;: \u0026amp;#34;#E06C75\u0026amp;#34;,\n            \u0026amp;#34;selectionBackground\u0026amp;#34;: \u0026amp;#34;#FFFFFF\u0026amp;#34;,\n            \u0026amp;#34;white\u0026amp;#34;: \u0026amp;#34;#ABB2BF\u0026amp;#34;,\n            \u0026amp;#34;yellow\u0026amp;#34;: \u0026amp;#34;#D19A66\u0026amp;#34;\n        }\n    ],\n    \u0026amp;#34;themes\u0026amp;#34;: [],\n    \u0026amp;#34;useAcrylicInTabRow\u0026amp;#34;: true\u003c/code\u003e\u003cspan class=\"line-numbers-rows\" aria-hidden=\"true\"\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003cspan\u003e\u003c/span\u003e\u003c/span\u003e\u003c/pre\u003e\u003ch2 id=\"addins-which-make-your-shell-look-and-function-even-better\"\u003eAddins which make your shell look and function even better\u003c/h2\u003e\n\u003cp\u003eBelow you will find some more links which are going to make your shell experience even better. It is a collection of tools i use daily. You can already look at the bookmarks and get an idea of what it does. I would recommend to read through every link and decide if this is also going to make your life easier.\u003c/p\u003e\n\u003cp\u003eI personally like Exa the most. This is a replacement for the \u0026rsquo;ls\u0026rsquo; command which creates a lot more visibility using colors and icons for folders and such. Also htop is something i already use for years as the goto process manager.\u003c/p\u003e\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://github.com/BurntSushi/ripgrep\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eGitHub - BurntSushi/ripgrep: ripgrep recursively searches directories for a regex pattern while respecting your gitignore\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eripgrep recursively searches directories for a regex pattern while respecting your gitignore - GitHub - BurntSushi/ripgrep: ripgrep recursively searches directories for a regex pattern while respec…\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/06-icon_hu_a8d8b8bb716d7df8.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eGitHub\u003c/span\u003e\u003cspan\u003eBurntSushi\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/06-thumb_hu_b211a5b1ef787f02.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://github.com/nvbn/thefuck\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eGitHub - nvbn/thefuck: Magnificent app which corrects your previous console command.\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eMagnificent app which corrects your previous console command. - GitHub - nvbn/thefuck: Magnificent app which corrects your previous console command.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/07-icon_hu_a8d8b8bb716d7df8.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eGitHub\u003c/span\u003e\u003cspan\u003envbn\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/07-thumb_hu_b2b48ab666833f4.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://github.com/rupa/z\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eGitHub - rupa/z: z - jump around\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003ez - jump around. Contribute to rupa/z development by creating an account on GitHub.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/08-icon_hu_a8d8b8bb716d7df8.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eGitHub\u003c/span\u003e\u003cspan\u003erupa\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/08-thumb_hu_4400b46cb8d5382e.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://github.com/ogham/exa\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003eGitHub - ogham/exa: A modern replacement for ‘ls’.\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003eA modern replacement for ‘ls’. Contribute to ogham/exa development by creating an account on GitHub.\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" sizes=\"25px\"\u003e\u003csource type=\"image/webp\" sizes=\"25px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/09-icon_hu_a8d8b8bb716d7df8.webp\" alt=\"\" class=\"bookmark__icon\" width=\"50\" height=\"50\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003cspan\u003eGitHub\u003c/span\u003e\u003cspan\u003eogham\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/09-thumb_hu_67f6b8479bab6f37.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n\u003cfigure class=\"bookmark\"\u003e\u003ca class=\"bookmark__link\" href=\"https://htop.dev/\" rel=\"noopener\"\u003e\u003cspan class=\"bookmark__body\"\u003e\u003cspan class=\"bookmark__title\"\u003ehtop - an interactive process viewer\u003c/span\u003e\u003cspan class=\"bookmark__description\"\u003e\u003c/span\u003e\u003cspan class=\"bookmark__meta\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/10-icon.ico\" sizes=\"25px\" alt=\"\" class=\"bookmark__icon\" loading=\"lazy\" decoding=\"async\"\u003e\u003cspan\u003ean interactive process viewer\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"bookmark__thumb\"\u003e\u003cpicture\u003e\u003csource type=\"image/avif\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/webp\" media=\"(max-width: 47.99rem)\" sizes=\"calc(100vw - 2.5rem)\"\u003e\u003csource type=\"image/avif\" sizes=\"250px\"\u003e\u003csource type=\"image/webp\" sizes=\"250px\"\u003e\u003cimg src=\"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/bookmarks/10-thumb_hu_745f32083f045968.webp\" alt=\"\" width=\"500\" height=\"435\" loading=\"lazy\" decoding=\"async\"\u003e\n  \u003c/picture\u003e\u003c/span\u003e\u003c/a\u003e\u003c/figure\u003e\n\n","date_modified":"2023-03-20T11:13:48+01:00","date_published":"2023-02-08T10:57:00+01:00","id":"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/","language":"en","summary":"In this post i will explain my particular setup of how i use my windows terminal along with some other tips. So if you want to impress someone while working in your favorite coffee place by making them feel like you work for Nasa.","tags":["Under the hood"],"title":"The ultimate ZSH and Windows Terminal look and feel","url":"https://www.allsubjectsmatter.nl/the-ultimate-zsh-and-windows-terminal-look-and-feel/"}],"language":"en","title":"All Subjects Matter","version":"https://jsonfeed.org/version/1.1"}