Merge feature/source-kinds into ticket/3-fun-section (#3 )

Keeps the Learning catalog from #6 and puts the Fun group after it, in the catalog and the README. Adds the end-to-end check #4 left for Fun: a custom Fun feed added on the Sources tab shows up in the Learn tab's Fun section, not in Today. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the Fun section to the Learn tab with comic-friendly reader view (#3 )
2026-10-16 15:58:17 +02:00 · 2026-10-16 15:56:42 +02:00
16 changed files with 420 additions and 159 deletions
@@ -25,9 +25,7 @@ or write a daily briefing. It's meant to replace the n8n → Discord workflow.
 - **Paid articles and ads hidden**: heise+ and Golem `(g+)` are recognized by
  their title; SPIEGEL+ and t3n Pro by a marker in the article page, which the app
  checks once per new article in the background (it reads only until the marker,
-  and remembers results for 14 days). Paid-subscriber posts from Substack newsletters
-  (Interconnects, Ahead of AI, …) are recognized in the feed itself, where Substack
-  cuts them off with a "Read more" link. Sponsored posts (`Anzeige:`) are hidden too.
+  and remembers results for 14 days). Sponsored posts (`Anzeige:`) are hidden too.
  The feed header shows how many were hidden; both filters can be turned off.
 - **Ad and cookie-banner blocking** in the original-page view: requests to ~80
  ad/tracker hosts and the common consent managers (Sourcepoint, OneTrust,
@@ -216,6 +214,21 @@ Thing (applied AI), vendor blogs (OpenAI, DeepMind, Hugging Face) and Chip Huyen
 (inactive since January 2025). Some feeds are long (Dan Luu ~130 posts, Lilian
 Weng ~50); the 14-day window keeps the list short.

+## Fun sources
+
+Tech humour for the Fun section of the Learn tab, which reaches back 3 days. Fun
+articles never show up in Today or the Briefing. ★ = enabled by default.
+
+| | Source | Feed | Focus |
+|---|---|---|---|
+| 🇬🇧 ★ | xkcd | `xkcd.com/atom.xml` | Webcomic of romance, sarcasm, math and language |
+| 🇬🇧 ★ | The Daily WTF | `thedailywtf.com/rss` | IT horror stories, bad code and error messages |
+| 🇬🇧 | r/ProgrammerHumor | `reddit.com/r/ProgrammerHumor/top/.rss?t=day` | The day's top programming memes on Reddit |
+
+Reader view shows comics and memes whole instead of as a cropped banner: xkcd
+with its title text underneath, The Daily WTF's Error'd with its screenshots.
+Reddit blocks reader view, so a meme shows the picture from the feed.
+
 ## Project layout

 ```
@@ -361,6 +361,37 @@ const defaultSources = <NewsSource>[
    kind: SourceKind.learning,
    description: 'Software architecture and engineering talks, high volume',
  ),
+
+  // --- Fun ----------------------------------------------------------------
+  NewsSource(
+    id: 'xkcd',
+    name: 'xkcd',
+    feedUrl: 'https://xkcd.com/atom.xml',
+    homepage: 'https://xkcd.com',
+    language: SourceLanguage.en,
+    kind: SourceKind.fun,
+    description: 'Webcomic of romance, sarcasm, math and language',
+    enabledByDefault: true,
+  ),
+  NewsSource(
+    id: 'thedailywtf',
+    name: 'The Daily WTF',
+    feedUrl: 'https://thedailywtf.com/rss',
+    homepage: 'https://thedailywtf.com',
+    language: SourceLanguage.en,
+    kind: SourceKind.fun,
+    description: 'IT horror stories, bad code and error messages',
+    enabledByDefault: true,
+  ),
+  NewsSource(
+    id: 'programmerhumor',
+    name: 'r/ProgrammerHumor',
+    feedUrl: 'https://www.reddit.com/r/ProgrammerHumor/top/.rss?t=day',
+    homepage: 'https://www.reddit.com/r/ProgrammerHumor/',
+    language: SourceLanguage.en,
+    kind: SourceKind.fun,
+    description: "The day's top programming memes on Reddit",
+  ),
 ];

 /// The curated sources before the app remembered which ones it had seen.
@@ -49,9 +49,10 @@ class ArticleExtractor {
  /// Hard cap on what we send to the model per article (~10k tokens).
  static const maxChars = 40000;

-  /// Loads the article page and returns whichever is longer: the extracted
-  /// page or the full content some feeds ship. Never throws; on failure
-  /// (paywall, bot protection, CORS on web) it falls back to feed content.
+  /// Loads the article page and returns whichever has more text (on a tie,
+  /// more blocks): the extracted page or the full content some feeds ship.
+  /// Never throws; on failure (paywall, bot protection, CORS on web) it falls
+  /// back to feed content.
  ///
  /// With [acceptConsentWalls], a redirect to a consent page the
  /// [ConsentService] knows is accepted once and the article fetched again.
@@ -81,7 +82,12 @@ class ArticleExtractor {
      if (page.wallUrl != null) return ArticleContent(fromFeed.blocks, consentWall: true);
      if (page.status != 200) return fromFeed;
      final fromPage = parse(FeedService.decodeBody(page.bytes), base);
-      return fromPage.plainText.length > fromFeed.plainText.length ? fromPage : fromFeed;
+      final pageText = fromPage.plainText.length, feedText = fromFeed.plainText.length;
+      // Without text either way (a comic), the one with the picture wins.
+      return pageText > feedText ||
+              (pageText == feedText && fromPage.blocks.length > fromFeed.blocks.length)
+          ? fromPage
+          : fromFeed;
    } catch (_) {
      return fromFeed;
    }
@@ -119,7 +125,8 @@ class ArticleExtractor {
  /// Class/id fragments of page furniture that sits inside article bodies.
  static final _junkMarker = RegExp(
    r'comment|related|newsletter|share|social|promo|advert|paywall|subscribe|'
-    r'recommend|author-box|byline|breadcrumb|tags|toolbar|consent|cookie|card|native-ad',
+    r'recommend|author-box|byline|breadcrumb|tags|toolbar|consent|cookie|card|native-ad|'
+    r'license',
    caseSensitive: false,
  );

@@ -155,14 +162,15 @@ class ArticleExtractor {
    final blocks = <ReaderBlock>[];
    final seenImages = <String>{};
    var chars = 0;
-    void add(BlockType type, String text) {
-      if (chars > maxChars) return;
+    bool add(BlockType type, String text) {
+      if (chars > maxChars) return false;
      if (type == BlockType.image) {
-        if (!seenImages.add(text)) return;
+        if (!seenImages.add(text)) return false;
      } else {
        chars += text.length;
      }
      blocks.add(ReaderBlock(type, text));
+      return true;
    }

    void walk(dom.Element el) {
@@ -190,6 +198,8 @@ class ArticleExtractor {
        case 'blockquote':
          final t = _clean(el.text);
          if (t.isNotEmpty) add(BlockType.quote, t);
+          // Quoted screenshots, as in The Daily WTF's Error'd.
+          el.querySelectorAll('img').forEach((img) => _addImage(img, base, add));
          return;
        case 'pre':
          final t = el.text.trimRight();
@@ -221,7 +231,7 @@ class ArticleExtractor {
  static String _clean(String s) => s.replaceAll(RegExp(r'\s+'), ' ').trim();

  static void _addImage(
-      dom.Element img, Uri base, void Function(BlockType, String) add) {
+      dom.Element img, Uri base, bool Function(BlockType, String) add) {
    // Lazy-loading sites keep the real URL in data attributes.
    final raw = img.attributes['data-src'] ??
        img.attributes['data-lazy-src'] ??
@@ -232,19 +242,25 @@ class ArticleExtractor {
    }
    final width = int.tryParse(img.attributes['width'] ?? '');
    if (width != null && width < 120) return; // icons, avatars, tracking pixels
+    if (img.attributes.containsKey('usemap')) return; // clickable navigation
    // Images linking to another page are author avatars or related-story
-    // thumbnails; links to the full-size image file are fine.
+    // thumbnails; links to the full-size image file or within the page are fine.
    for (var p = img.parent; p != null; p = p.parent) {
      if (p.localName == 'a') {
        final href = p.attributes['href'] ?? '';
-        if (!RegExp(r'\.(jpe?g|png|webp|gif|avif)(\?|$)', caseSensitive: false)
-            .hasMatch(href)) {
+        if (!href.startsWith('#') &&
+            !RegExp(r'\.(jpe?g|png|webp|gif|avif)(\?|$)', caseSensitive: false).hasMatch(href)) {
          return;
        }
        break;
      }
    }
    final url = base.resolve(raw).toString();
-    if (url.startsWith('http')) add(BlockType.image, url);
+    if (!url.startsWith('http') || !add(BlockType.image, url)) return;
+    // Comics such as xkcd put a last joke in the title text.
+    final title = _clean(img.attributes['title'] ?? '');
+    if (title.length >= 20 && title != _clean(img.attributes['alt'] ?? '')) {
+      add(BlockType.caption, title);
+    }
  }
 }
@@ -9,8 +9,6 @@ import 'storage.dart';
 /// Recognizes paid and sponsored articles so the feed can hide them.
 ///
 /// Most paid articles are marked in the title (heise+, Golem's "(g+)").
-/// Substack newsletters cut paid posts short in the feed and end them with a
-/// "Read more" link to the post itself, so those are recognized from the feed.
 /// SPIEGEL and t3n only mark them in the page's schema.org data
 /// (`"isAccessibleForFree": false`), so for those sources each new article is
 /// checked once in the background and the result is remembered.
@@ -35,14 +33,6 @@ class PaywallService {
    caseSensitive: false,
  );

-  /// What Substack appends to a paid post's feed content in place of the rest:
-  /// `<p><a href="{post url}">Read more</a></p>`, as the very end. Free posts
-  /// end with their own text, even when they ask readers to subscribe.
-  static final _substackCut = RegExp(
-    r'<p>\s*<a href="([^"]+)">\s*Read more\s*</a>\s*</p>\s*$',
-    caseSensitive: false,
-  );
-
  static final _marker =
      RegExp(r'''"isAccessibleForFree"\s*:\s*"?(true|false)"?''', caseSensitive: false);

@@ -51,11 +41,7 @@ class PaywallService {
  static const _concurrency = 6;
  static const _keepFor = Duration(days: 14);

-  bool isPaid(Article a) =>
-      paidTitle.hasMatch(a.title) || _isCutSubstackPost(a) || (_checks[a.link]?.paid ?? false);
-
-  static bool _isCutSubstackPost(Article a) =>
-      _substackCut.firstMatch(a.contentHtml)?.group(1) == a.link;
+  bool isPaid(Article a) => paidTitle.hasMatch(a.title) || (_checks[a.link]?.paid ?? false);

  bool isSponsored(Article a) => sponsoredTitle.hasMatch(a.title);

@@ -6,6 +6,7 @@ import 'package:flutter/material.dart';
 import 'package:flutter_inappwebview/flutter_inappwebview.dart';

 import '../models/article.dart';
+import '../models/news_source.dart';
 import '../services/article_extractor.dart';
 import '../services/consent_service.dart';
 import '../services/content_blocking.dart';
@@ -275,12 +276,22 @@ class _ReaderView extends StatelessWidget {
        builder: (context, snapshot) {
          final loaded = snapshot.data;
          var blocks = loaded?.blocks ?? const <ReaderBlock>[];
+          // Comics and memes are the picture: show it whole in the body
+          // rather than as a cropped banner.
+          final pictureLed = article.source.kind == SourceKind.fun;
+          final banner = pictureLed ? null : article.imageUrl;
          // The page's own hero image duplicates the feed image shown on top.
-          if (article.imageUrl != null) {
+          if (banner != null) {
            blocks = blocks.skipWhile((b) => b.type == BlockType.image).toList();
          }
+          if (pictureLed &&
+              article.imageUrl != null &&
+              !blocks.any((b) => b.type == BlockType.image)) {
+            blocks = [ReaderBlock(BlockType.image, article.imageUrl!), ...blocks];
+          }
          final isFull = loaded != null &&
-              loaded.plainText.length >= max(400, article.excerpt.length * 2);
+              (loaded.plainText.length >= max(400, article.excerpt.length * 2) ||
+                  (pictureLed && blocks.any((b) => b.type == BlockType.image)));
          // Skip the lead if the body already starts with it.
          final firstText = blocks.where((b) => b.type == BlockType.paragraph).firstOrNull;
          final showLead = article.excerpt.isNotEmpty &&
@@ -291,9 +302,9 @@ class _ReaderView extends StatelessWidget {
          return ListView(
            padding: const EdgeInsets.only(bottom: 48),
            children: [
-              if (article.imageUrl != null)
+              if (banner != null)
                Image.network(
-                  article.imageUrl!,
+                  banner,
                  fit: BoxFit.cover,
                  webHtmlElementStrategy: WebHtmlElementStrategy.prefer,
                  // Take up space only once the image has loaded, so a broken
@@ -336,7 +347,7 @@ class _ReaderView extends StatelessWidget {
                      ),
                    if (isFull)
                      for (final block in blocks)
-                        if (block.text != article.imageUrl) _BlockView(block: block),
+                        if (block.text != banner) _BlockView(block: block),
                    if (loaded != null) ...[
                      const SizedBox(height: 24),
                      const Divider(),
@@ -13,7 +13,7 @@ class LearnScreen extends StatelessWidget {
  final AppState state;

  /// The sections, top to bottom.
-  static const kinds = [SourceKind.learning];
+  static const kinds = [SourceKind.learning, SourceKind.fun];

  @override
  Widget build(BuildContext context) {
@@ -102,8 +102,86 @@ void main() {
        isFragment: true);
    expect(content.blocks, hasLength(2));
  });
+
+  test('an xkcd page gives the comic and its title text, without the page around it', () {
+    final content = ArticleExtractor.parse(xkcdPage, Uri.parse('https://xkcd.com/3312/'));
+
+    expect(content.blocks.map((b) => (b.type, b.text)), [
+      (BlockType.image, 'https://imgs.xkcd.com/comics/momentum_demonstration.png'),
+      (BlockType.caption, xkcdTitleText),
+    ]);
+  });
+
+  test("The Daily WTF's Error'd screenshots show between their stories", () {
+    final content = ArticleExtractor.parse(
+        errordPage, Uri.parse('https://thedailywtf.com/articles/scrollpocalypse'));
+
+    expect(content.blocks.map((b) => b.type), [
+      BlockType.paragraph,
+      BlockType.image,
+      BlockType.paragraph,
+      BlockType.image,
+    ]);
+    expect(content.blocks[1].text, 'https://cdn.example.com/errord/first.png');
+  });
 }

+const xkcdTitleText = '"I\'m doing really well on the Hertzsprung-Russell academic diagram, '
+    'which has spectral class on one axis and Rate My Professors rating on the other."';
+
+/// Trimmed from https://xkcd.com/3312/.
+const xkcdPage = '''
+<!DOCTYPE html>
+<html><head><title>xkcd: Momentum Demonstration</title></head>
+<body>
+<div id="topContainer">
+<div id="topLeft"><ul><li><a href="/archive">Archive</a></li><li><a href="/about">About</a></li></ul></div>
+<div id="topRight">
+<div id="masthead"><span><a href="/"><img src="/s/0b7742.png" alt="xkcd.com logo" height="83" width="185"/></a></span>
+<span id="slogan">A webcomic of romance,<br/> sarcasm, math, and language.</span></div>
+<div id="news"><div id="xkcdNews"><a href="https://store.xkcd.com/"><img src="https://imgs.xkcd.com/store_news/news_back.png" alt="The xkcd store is back, with shirts, stickers, and posters for sale." title="xkcd store news" /></a></div></div>
+</div>
+</div>
+<div id="middleContainer" class="box">
+<div id="ctitle">Momentum Demonstration</div>
+<ul class="comicNav"><li><a href="/1/">|&lt;</a></li><li><a rel="prev" href="/3311/">&lt; Prev</a></li></ul>
+<div id="comic">
+<img src="//imgs.xkcd.com/comics/momentum_demonstration.png" title="&quot;I&#39;m doing really well on the Hertzsprung-Russell academic diagram, which has spectral class on one axis and Rate My Professors rating on the other.&quot;" alt="Momentum Demonstration" srcset="//imgs.xkcd.com/comics/momentum_demonstration_2x.png 2x" style="image-orientation:none" />
+</div>
+<ul class="comicNav"><li><a href="/1/">|&lt;</a></li><li><a rel="prev" href="/3311/">&lt; Prev</a></li></ul>
+<br />
+Permanent link to this comic: <a href="https://xkcd.com/3312">https://xkcd.com/3312/</a><br />
+Image URL (for hotlinking/embedding): <a href= "https://imgs.xkcd.com/comics/momentum_demonstration.png">https://imgs.xkcd.com/comics/momentum_demonstration.png</a>
+<div id="transcript" style="display: none"></div>
+</div>
+<div id="bottom" class="box">
+<img src="//imgs.xkcd.com/s/a899e84.jpg" width="520" height="100" alt="Selected Comics" usemap="#comicmap"/>
+<map id="comicmap" name="comicmap"><area shape="rect" coords="0,0,100,100" href="/150/" alt="Grownups"/></map>
+<div id="comicLinks">Comics I enjoy:<br/><a href="http://threewordphrase.com/">Three Word Phrase</a></div>
+<div id="footnote" style="width:70%">xkcd.com is best viewed with Netscape Navigator 4.0 or below on a Pentium 3&plusmn;1 emulated in Javascript on an Apple IIGS</div>
+<div id="licenseText">
+<p>
+This work is licensed under a
+<a href="https://creativecommons.org/licenses/by-nc/2.5/">Creative Commons Attribution-NonCommercial 2.5 License</a>.
+</p><p>
+This means you're free to copy and share these comics (but not to sell them). <a rel="license" href="/license.html">More details</a>.</p>
+</div>
+</div>
+</body></html>''';
+
+/// Trimmed from https://thedailywtf.com/articles/scrollpocalypse.
+const errordPage = '''
+<html><body>
+<div itemscope itemtype="http://schema.org/Article" class="article-container errord">
+<div itemprop="articleBody" class="article-body">
+<p>Dr. Doom...scroller titled this one "Scrollbar-ception". Said he "Screenshot of the TikTok webpage."</p>
+<blockquote><p><a href="#d6711f9e"><img itemprop="image" border="0" alt="d6711f9e" src="https://cdn.example.com/errord/first.png"/></a></p><p> </p></blockquote>
+<p>An anonymous reader clarified "I was reorganizing the document by drag-and-dropping table cells."</p>
+<blockquote><p><a href="#01270f37"><img itemprop="image" border="0" alt="01270f37" src="https://cdn.example.com/errord/second.png"/></a></p><p> </p></blockquote>
+</div>
+</div>
+</body></html>''';
+
 /// Answers every request as if the server had redirected to [finalUrl].
 class _RedirectingClient extends http.BaseClient {
  _RedirectingClient(this.finalUrl, this.body);
@@ -0,0 +1,109 @@
+import 'dart:io';
+import 'dart:ui' as ui;
+
+import 'package:daily_tech_news/models/article.dart';
+import 'package:daily_tech_news/models/news_source.dart';
+import 'package:daily_tech_news/services/article_extractor.dart';
+import 'package:daily_tech_news/ui/article_screen.dart';
+import 'package:flutter/material.dart';
+import 'package:flutter_test/flutter_test.dart';
+import 'package:http/http.dart' as http;
+import 'package:http/testing.dart';
+import 'package:shared_preferences/shared_preferences.dart';
+
+import 'support/fake_app.dart';
+
+/// A plain PNG of the given size, standing in for a comic strip or meme.
+Future<List<int>> _png(int width, int height) async {
+  final recorder = ui.PictureRecorder();
+  Canvas(recorder).drawRect(
+      Rect.fromLTWH(0, 0, width.toDouble(), height.toDouble()), Paint()..color = Colors.black);
+  final image = await recorder.endRecording().toImage(width, height);
+  return (await image.toByteData(format: ui.ImageByteFormat.png))!.buffer.asUint8List();
+}
+
+/// Serves a [width] x [height] PNG from this machine, so the reader view can
+/// load and lay out a real picture. Returns its URL.
+Future<String> _serveImage(WidgetTester tester, int width, int height) async {
+  HttpOverrides.global = null;
+  final server = (await tester.runAsync(() => HttpServer.bind(InternetAddress.loopbackIPv4, 0)))!;
+  addTearDown(() => tester.runAsync(server.close));
+  final png = (await tester.runAsync(() => _png(width, height)))!;
+  server.listen((request) => request.response
+    // No idle keep-alive timer left behind in the test's fake time.
+    ..persistentConnection = false
+    ..headers.contentType = ContentType('image', 'png')
+    ..add(png)
+    ..close());
+  return 'http://127.0.0.1:${server.port}/picture.png';
+}
+
+/// Opens [article] in the reader, with the article page answered by [page],
+/// and waits until the picture at [imageUrl] has loaded. Returns its finder.
+Future<Finder> _openAndLoad(
+    WidgetTester tester, Article article, String imageUrl, http.Response page) async {
+  final state = (await tester.runAsync(() => launchApp([article.source],
+      extractor: ArticleExtractor(client: MockClient((_) async => page)))))!;
+  await tester.pumpWidget(MaterialApp(home: ArticleScreen(state: state, article: article)));
+  final picture = find.descendant(
+      of: find.byWidgetPredicate((w) =>
+          w is Image && w.image is NetworkImage && (w.image as NetworkImage).url == imageUrl),
+      matching: find.byType(RawImage));
+  for (var i = 0; i < 100; i++) {
+    await tester.runAsync(() => Future<void>.delayed(const Duration(milliseconds: 20)));
+    await tester.pump();
+    if (picture.evaluate().isNotEmpty && tester.widget<RawImage>(picture).image != null) break;
+  }
+  return picture;
+}
+
+double _aspect(WidgetTester tester, Finder picture) {
+  final size = tester.getSize(picture);
+  return size.height / size.width;
+}
+
+const _titleText = 'The title text is the last joke, hidden in the mouse-over.';
+
+void main() {
+  setUp(() => SharedPreferences.setMockInitialValues({}));
+
+  testWidgets('opening an xkcd comic shows the whole comic and its title text', (tester) async {
+    final comicUrl = await _serveImage(tester, 300, 100);
+    final page = '''
+<html><body>
+<div id="ctitle">Strip</div>
+<div id="comic"><img src="$comicUrl" title="$_titleText" alt="Strip"></div>
+<div id="licenseText"><p>This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License.</p></div>
+</body></html>''';
+    // What the xkcd feed gives: a title, a link and the comic as the image.
+    final article = Article(
+        title: 'Strip',
+        link: 'https://xkcd.com/9999/',
+        source: testSource('xkcd', SourceKind.fun),
+        imageUrl: comicUrl);
+
+    final comic = await _openAndLoad(tester, article, comicUrl,
+        http.Response(page, 200, headers: {'content-type': 'text/html; charset=utf-8'}));
+
+    expect(tester.widget<RawImage>(comic).image, isNotNull, reason: 'the comic loads');
+    expect(_aspect(tester, comic), closeTo(100 / 300, 0.01), reason: 'the comic is not cropped');
+    expect(find.text(_titleText), findsOneWidget);
+    expect(find.textContaining("couldn't be loaded"), findsNothing);
+  });
+
+  testWidgets('a meme whose page blocks the reader still shows the picture from the feed',
+      (tester) async {
+    final memeUrl = await _serveImage(tester, 100, 200);
+    final article = Article(
+        title: 'vibeCoding',
+        link: 'https://www.reddit.com/r/ProgrammerHumor/comments/1/vibecoding/',
+        source: testSource('programmerhumor', SourceKind.fun),
+        imageUrl: memeUrl);
+
+    final meme = await _openAndLoad(tester, article, memeUrl, http.Response('Blocked', 403));
+
+    expect(tester.widget<RawImage>(meme).image, isNotNull, reason: 'the meme loads');
+    expect(_aspect(tester, meme), closeTo(200 / 100, 0.01), reason: 'the meme is not cropped');
+    expect(find.textContaining("couldn't be loaded"), findsNothing);
+  });
+}
@@ -1,15 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Ahead of AI]]></title><link>https://magazine.sebastianraschka.com</link><generator>Substack</generator>
-<!-- Real items from https://magazine.sebastianraschka.com/feed (October 2026), content:encoded trimmed to its first paragraph and its end. -->
-<!-- Paid post (audience "only_paid" on its page): Substack cuts the content and ends it with a "Read more" link to the post. -->
-<item><title><![CDATA[Building an AI Text Detector From Scratch]]></title><description><![CDATA[An End-to-End Project With Dataset Construction, Model Training, Local Deployment, and RLVR]]></description><link>https://magazine.sebastianraschka.com/p/ai-detector-from-scratch</link><guid isPermaLink="false">https://magazine.sebastianraschka.com/p/ai-detector-from-scratch</guid><dc:creator><![CDATA[Sebastian Raschka, PhD]]></dc:creator><pubDate>Sat, 15 Aug 2026 11:54:24 GMT</pubDate><content:encoded><![CDATA[<p>Substack recently launched its AI detector feature in the UI, which is super interesting.</p><p>For this, we are going to fine-tune a DistilBERT classifier (similar to what I described in one of my early Substack articles, <a href="https://magazine.sebastianraschka.com/p/finetuning-large-language-models">Finetuning Large Language Models</a>), but more details on that later when we get to that stage.</p><p></p>
-      <p>
-          <a href="https://magazine.sebastianraschka.com/p/ai-detector-from-scratch">
-              Read more
-          </a>
-      </p>
-   ]]></content:encoded></item>
-<!-- Free post (audience "everyone") whose footer asks readers to become paid subscribers. -->
-<item><title><![CDATA[Controlling Reasoning Effort in LLMs]]></title><description><![CDATA[How LLMs Learn Low-, Medium-, and High-Effort Reasoning Modes]]></description><link>https://magazine.sebastianraschka.com/p/controlling-reasoning-effort-in-llms</link><guid isPermaLink="false">https://magazine.sebastianraschka.com/p/controlling-reasoning-effort-in-llms</guid><dc:creator><![CDATA[Sebastian Raschka, PhD]]></dc:creator><pubDate>Sat, 18 Jul 2026 11:16:09 GMT</pubDate><content:encoded><![CDATA[<p><span>It has been almost two years since OpenAI released o1, a model that popularized the idea of LLM-based reasoning models. DeepSeek-R1 followed about four months later, together with details of a reinforcement learning with verifiable rewards (RLVR) recipe to train such reasoning models.</span></p><p>If you liked my previous <a href="https://amzn.to/4fqvn0D">Build a Large Language Model (From Scratch)</a> book, this is essentially a sequel implementing inference-time scaling techniques and reinforcement learning algorithms from scratch.</p><p>And if you want to support future long-form articles like this one, consider <a href="https://magazine.sebastianraschka.com/subscribe">becoming a paid subscriber</a>. It helps me keep writing these independent deep dives and sharing the accompanying code, figures, and experiments.</p><p></p>]]></content:encoded></item>
-<!-- Free post (audience "everyone") that ends with Substack's subscribe widget. -->
-<item><title><![CDATA[GPT-6 Astra, Looped Transformers, and Hidden Reasoning]]></title><description><![CDATA[A Look at Recurrent Depth, Hidden Chains of Thought, and Recent Research on Looping Transformer Blocks]]></description><link>https://magazine.sebastianraschka.com/p/gpt-6-astra-looped-transformers-and</link><guid isPermaLink="false">https://magazine.sebastianraschka.com/p/gpt-6-astra-looped-transformers-and</guid><dc:creator><![CDATA[Sebastian Raschka, PhD]]></dc:creator><pubDate>Wed, 09 Sep 2026 11:14:25 GMT</pubDate><content:encoded><![CDATA[<p>A lot has happened in the last few weeks. I am sure that OpenAI&#8217;s GPT-6 Astra is top of mind for everyone right now. In particular, thoughts on its performance, the looped transformer/recurrent depth aspects, and rumors that Astra is &#8220;hiding&#8221; its reasoning trace (i.e., chain of thought).</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://magazine.sebastianraschka.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption"></p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item>
-</channel></rss>
@@ -1,13 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Interconnects AI]]></title><link>https://www.interconnects.ai</link><generator>Substack</generator>
-<!-- Real items from https://www.interconnects.ai/feed (October 2026), content:encoded trimmed to its first paragraph and its end. -->
-<!-- Paid post (audience "only_paid" on its page). -->
-<item><title><![CDATA[Lessons from the hacks]]></title><description><![CDATA[Musings on model alignment, what determines safety, and where we go from here.]]></description><link>https://www.interconnects.ai/p/lessons-from-the-hacks</link><guid isPermaLink="false">https://www.interconnects.ai/p/lessons-from-the-hacks</guid><dc:creator><![CDATA[Nathan Lambert]]></dc:creator><pubDate>Sun, 09 Aug 2026 14:57:11 GMT</pubDate><content:encoded><![CDATA[<p><span>The recent run of cyberattacks by in-development frontier models has got me thinking a lot about how our current incentive systems are not well suited for such fast technological transitions.</span></p><div><hr></div><h4>Book sale</h4><p><span>I&#8217;m also hosting a book launch where you can get a free signed copy tomorrow from 5-8PM in Seattle (Fremont/Ballard area) &#8211; we still have some extra space so I&#8217;m opening signups to paid subscribers below the paywall:</span></p>
-      <p>
-          <a href="https://www.interconnects.ai/p/lessons-from-the-hacks">
-              Read more
-          </a>
-      </p>
-   ]]></content:encoded></item>
-<!-- Free post (audience "everyone") that mentions paid subscribers and ends with an embedded post's "Read more". -->
-<item><title><![CDATA[When will average people feel AI’s impact?]]></title><description><![CDATA[We&#8217;re <5 years into a compounding revolution which could take a century, and how the AI industry should manage this.]]></description><link>https://www.interconnects.ai/p/when-will-average-people-feel-ais</link><guid isPermaLink="false">https://www.interconnects.ai/p/when-will-average-people-feel-ais</guid><dc:creator><![CDATA[Nathan Lambert]]></dc:creator><pubDate>Wed, 09 Sep 2026 11:01:23 GMT</pubDate><content:encoded><![CDATA[<h5>Housekeeping: Paid subscribers to Interconnects now get a permanent 40% discount on my book when purchasing at Manning.com. Access the code at the Interconnects <a href="https://www.interconnects.ai/perks">perks page</a>.</h5><p>Many AI optimists tend to compare what is happening in this AI boom to the industrial revolution, or to other periods of rapid technological advancement and diffusion into society.</p><div><a class="embedded-post" native="true" href="https://www.fabricatedknowledge.com/p/mythos-and-engels-pause?utm_source=substack&amp;utm_campaign=post_embed&amp;utm_medium=web&amp;embedding_publication_id=48206"><div class="embedded-post-header"><span class="embedded-post-publication-name">Fabricated Knowledge</span></div><div class="embedded-post-title-wrapper"><div class="embedded-post-title">Engels' Pause and the Permanent Underclass </div></div><div class="embedded-post-cta-wrapper"><span class="embedded-post-cta">Read more</span></div><div class="embedded-post-meta">6 months ago &#183; 214 likes &#183; 5 comments &#183; Doug O'Laughlin</div></a></div>]]></content:encoded></item>
-</channel></rss>
@@ -0,0 +1,102 @@
+import 'package:daily_tech_news/data/default_sources.dart';
+import 'package:daily_tech_news/models/article.dart';
+import 'package:daily_tech_news/models/briefing.dart';
+import 'package:daily_tech_news/models/news_source.dart';
+import 'package:daily_tech_news/services/summarizer.dart';
+import 'package:daily_tech_news/state/app_state.dart';
+import 'package:daily_tech_news/ui/feed_screen.dart';
+import 'package:daily_tech_news/ui/learn_screen.dart';
+import 'package:flutter/material.dart';
+import 'package:flutter_test/flutter_test.dart';
+import 'package:shared_preferences/shared_preferences.dart';
+
+import 'support/fake_app.dart';
+
+/// Remembers which articles a Briefing was asked for.
+class _RecordingSummarizer implements Summarizer {
+  List<Article>? briefingInput;
+
+  @override
+  String get name => 'test';
+
+  @override
+  Future<String> summarizeArticle(Article article, String text, SourceLanguage language) async =>
+      '';
+
+  @override
+  Future<DailyBriefing> createBriefing(List<Article> articles, SourceLanguage language) async {
+    briefingInput = articles;
+    return DailyBriefing(createdAt: DateTime.now(), intro: '', topics: const [], generatedBy: name);
+  }
+}
+
+void main() {
+  final verge = testSource('verge', SourceKind.news);
+  final bytebytego = testSource('bytebytego', SourceKind.learning);
+  final xkcd = testSource('xkcd', SourceKind.fun);
+  final feeds = {
+    verge.feedUrl: [('News from this morning', const Duration(hours: 2))],
+    bytebytego.feedUrl: [('Explainer from 2 days ago', const Duration(days: 2))],
+    xkcd.feedUrl: [('Comic from today', const Duration(hours: 5)),
+      ('Comic from 2 days ago', const Duration(days: 2)),
+      ('Comic from 4 days ago', const Duration(days: 4))],
+  };
+
+  setUp(() => SharedPreferences.setMockInitialValues({}));
+
+  Future<AppState> launch(WidgetTester tester, {Summarizer? summarizer}) async =>
+      (await tester.runAsync(() => launchApp([verge, bytebytego, xkcd],
+          feeds: feeds, summarizer: summarizer)))!;
+
+  double top(WidgetTester tester, String text) => tester.getTopLeft(find.text(text)).dy;
+
+  test('the curated Fun sources are xkcd and The Daily WTF on, r/ProgrammerHumor off', () {
+    final fun = {
+      for (final s in defaultSources)
+        if (s.kind == SourceKind.fun) s.name: (s.feedUrl, s.language, s.enabledByDefault),
+    };
+
+    expect(fun, {
+      'xkcd': ('https://xkcd.com/atom.xml', SourceLanguage.en, true),
+      'The Daily WTF': ('https://thedailywtf.com/rss', SourceLanguage.en, true),
+      'r/ProgrammerHumor': (
+        'https://www.reddit.com/r/ProgrammerHumor/top/.rss?t=day',
+        SourceLanguage.en,
+        false,
+      ),
+    });
+  });
+
+  testWidgets('the Learn tab has a Fun section below Learning with the last 3 days of Fun',
+      (tester) async {
+    final state = await launch(tester);
+
+    await tester.pumpWidget(MaterialApp(home: LearnScreen(state: state)));
+
+    expect(find.text('Fun'), findsOneWidget);
+    expect(find.text('2 articles · last 3 days'), findsOneWidget);
+    expect(top(tester, 'Learning'), lessThan(top(tester, 'Fun')));
+    expect(top(tester, 'Explainer from 2 days ago'), lessThan(top(tester, 'Fun')));
+    expect(top(tester, 'Fun'), lessThan(top(tester, 'Comic from today')));
+    expect(top(tester, 'Comic from today'), lessThan(top(tester, 'Comic from 2 days ago')));
+    expect(find.text('Comic from 4 days ago'), findsNothing);
+  });
+
+  testWidgets('Today leaves out Fun articles', (tester) async {
+    final state = await launch(tester);
+
+    await tester.pumpWidget(MaterialApp(home: FeedScreen(state: state)));
+
+    expect(find.text('News from this morning'), findsOneWidget);
+    expect(find.text('Comic from today'), findsNothing);
+  });
+
+  testWidgets('the Briefing leaves out Fun articles', (tester) async {
+    final summarizer = _RecordingSummarizer();
+    final state = await launch(tester, summarizer: summarizer);
+
+    await tester.runAsync(state.generateBriefing);
+
+    expect(summarizer.briefingInput!.map((a) => a.title), ['News from this morning']);
+  });
+}
@@ -1,75 +0,0 @@
-import 'dart:io';
-
-import 'package:daily_tech_news/models/article.dart';
-import 'package:daily_tech_news/models/news_source.dart';
-import 'package:daily_tech_news/services/feed_parser.dart';
-import 'package:daily_tech_news/services/feed_service.dart';
-import 'package:flutter_test/flutter_test.dart';
-import 'package:shared_preferences/shared_preferences.dart';
-
-import 'support/fake_app.dart';
-
-/// Serves saved Substack feeds (trimmed real items) instead of the network,
-/// dated an hour ago so every time window keeps them.
-class _SavedFeeds extends FeedService {
-  _SavedFeeds(this.fixtureByFeedUrl);
-
-  final Map<String, String> fixtureByFeedUrl;
-
-  @override
-  Future<List<Article>> fetchSource(NewsSource source, {String corsProxy = ''}) async {
-    final xml = File(fixtureByFeedUrl[source.feedUrl]!).readAsStringSync();
-    final anHourAgo = DateTime.now().toUtc().subtract(const Duration(hours: 1));
-    return [
-      for (final a in FeedParser.parse(xml, source))
-        Article(
-          title: a.title,
-          link: a.link,
-          source: a.source,
-          published: anHourAgo,
-          excerpt: a.excerpt,
-          contentHtml: a.contentHtml,
-        ),
-    ];
-  }
-}
-
-List<String> _titles(List<Article> articles) => articles.map((a) => a.title).toList();
-
-void main() {
-  final interconnects = testSource('interconnects', SourceKind.learning);
-  final substackNews = testSource('substack-news', SourceKind.news);
-
-  setUp(() => SharedPreferences.setMockInitialValues({}));
-
-  _SavedFeeds savedFeeds() => _SavedFeeds({
-        interconnects.feedUrl: 'test/fixtures/interconnects_feed.xml',
-        substackNews.feedUrl: 'test/fixtures/ahead_of_ai_feed.xml',
-      });
-
-  test('paid newsletter posts are hidden in the Learn tab and Today and counted as hidden',
-      () async {
-    final state = await launchApp([interconnects, substackNews], feedService: savedFeeds());
-
-    expect(_titles(state.articlesOf(SourceKind.learning)),
-        ['When will average people feel AI’s impact?']);
-    expect(state.hiddenCountOf(SourceKind.learning), 1);
-    expect(_titles(state.articlesOf(SourceKind.news)), [
-      'Controlling Reasoning Effort in LLMs',
-      'GPT-6 Astra, Looped Transformers, and Hidden Reasoning',
-    ]);
-    expect(state.hiddenCountOf(SourceKind.news), 1);
-  });
-
-  test('with "hide paid articles" off, paid newsletter posts show as before', () async {
-    final state = await launchApp([interconnects, substackNews], feedService: savedFeeds());
-
-    await state.updateSettings(state.settings.copyWith(hidePaid: false));
-
-    expect(_titles(state.articlesOf(SourceKind.learning)).toSet(),
-        {'Lessons from the hacks', 'When will average people feel AI’s impact?'});
-    expect(state.hiddenCountOf(SourceKind.learning), 0);
-    expect(_titles(state.articlesOf(SourceKind.news)), contains('Building an AI Text Detector From Scratch'));
-    expect(state.hiddenCountOf(SourceKind.news), 0);
-  });
-}
@@ -1,8 +1,5 @@
-import 'dart:io';
-
 import 'package:daily_tech_news/models/article.dart';
 import 'package:daily_tech_news/models/news_source.dart';
-import 'package:daily_tech_news/services/feed_parser.dart';
 import 'package:daily_tech_news/services/paywall_service.dart';
 import 'package:daily_tech_news/services/storage.dart';
 import 'package:flutter_test/flutter_test.dart';
@@ -41,21 +38,6 @@ void main() {
    expect(paywall.isSponsored(_article('Anzeigenmarkt bricht ein')), isFalse);
  });

-  test('recognizes paid Substack posts by the cut-off content in the feed', () async {
-    final paywall = PaywallService(await _storage(), client: MockClient((_) async => http.Response('', 500)));
-    final posts = {
-      for (final a in FeedParser.parse(
-          File('test/fixtures/ahead_of_ai_feed.xml').readAsStringSync(), _source()))
-        a.title: a,
-    };
-
-    expect(paywall.isPaid(posts['Building an AI Text Detector From Scratch']!), isTrue);
-    expect(paywall.isPaid(posts['Controlling Reasoning Effort in LLMs']!), isFalse,
-        reason: 'a free post asking readers to become paid subscribers');
-    expect(paywall.isPaid(posts['GPT-6 Astra, Looped Transformers, and Hidden Reasoning']!), isFalse,
-        reason: 'a free post ending with the subscribe widget');
-  });
-
  test('checks pages only for flagged sources and remembers results', () async {
    final storage = await _storage();
    final requested = <String>[];
@@ -95,6 +95,7 @@ void main() {
      'heise', 'verge', 'mit-tr', 'new-source',
      // Added to the catalog later, on by default.
      'techmeme', 'lobsters', 'the-decoder', 'simonwillison', ..._learningOnByDefault,
+      'xkcd', 'thedailywtf',
    });
  });

@@ -110,6 +111,7 @@ void main() {
    expect(state.enabledSourceIds, {
      'heise', 'golem', 't3n', 'verge',
      'techmeme', 'lobsters', 'the-decoder', 'simonwillison', ..._learningOnByDefault,
+      'xkcd', 'thedailywtf',
    });
    expect(state.enabledSources.map((s) => s.id), isNot(contains('wired')));
    expect(state.enabledSourceIds, isNot(contains('infoq')));
@@ -1,6 +1,7 @@
 import 'package:daily_tech_news/models/news_source.dart';
 import 'package:daily_tech_news/state/app_state.dart';
 import 'package:daily_tech_news/ui/home_shell.dart';
+import 'package:daily_tech_news/ui/learn_screen.dart';
 import 'package:daily_tech_news/ui/sources_screen.dart';
 import 'package:flutter/material.dart';
 import 'package:flutter_test/flutter_test.dart';
@@ -150,4 +151,34 @@ void main() {
    expect(find.text('News from this morning'), findsOneWidget);
    expect(find.text('Explainer from my blog'), findsNothing);
  });
+
+  testWidgets('a custom Fun source shows up in the Fun section of the Learn tab, not in Today',
+      (tester) async {
+    final feeds = FakeFeeds({
+      verge.feedUrl: [('News from this morning', const Duration(hours: 2))],
+      'https://example.com/mycomic.xml': [('Comic from my feed', const Duration(days: 1))],
+    });
+    final state = (await tester.runAsync(() => launchApp([verge], feedService: feeds)))!;
+    tester.view.physicalSize = const Size(800, 2400);
+    tester.view.devicePixelRatio = 1;
+    addTearDown(tester.view.reset);
+    await tester.pumpWidget(MaterialApp(home: HomeShell(state: state)));
+    Finder tab(String label) =>
+        find.descendant(of: find.byType(NavigationBar), matching: find.text(label));
+    Finder onLearn(String text) =>
+        find.descendant(of: find.byType(LearnScreen), matching: find.text(text));
+
+    await tester.tap(tab('Sources'));
+    await tester.pump();
+    await addFeed(tester, 'mycomic', kind: 'Fun');
+    await tester.tap(tab('Learn'));
+    await tester.pumpAndSettle();
+
+    expectInOrder(tester, [onLearn('Learning'), onLearn('Fun'), onLearn('Comic from my feed')]);
+
+    await tester.tap(tab('Today'));
+    await tester.pumpAndSettle();
+    expect(find.text('News from this morning'), findsOneWidget);
+    expect(find.text('Comic from my feed'), findsNothing);
+  });
 }
@@ -1,5 +1,6 @@
 import 'package:daily_tech_news/models/article.dart';
 import 'package:daily_tech_news/models/news_source.dart';
+import 'package:daily_tech_news/services/article_extractor.dart';
 import 'package:daily_tech_news/services/feed_service.dart';
 import 'package:daily_tech_news/services/storage.dart';
 import 'package:daily_tech_news/services/summarizer.dart';
@@ -58,11 +59,13 @@ Future<AppState> launchApp(
  Map<String, List<(String, Duration)>> feeds = const {},
  FeedService? feedService,
  Summarizer? summarizer,
+  ArticleExtractor? extractor,
 }) async {
  final state = AppState(await Storage.open(),
      feedService: feedService ?? FakeFeeds(feeds),
      curatedSources: curatedSources,
-      summarizer: summarizer);
+      summarizer: summarizer,
+      extractor: extractor);
  await state.init();
  return state;
 }