Syncing my Feedly Annotations to Obsidian

For several years I’ve been a loyal customer of Feedly, a company building a top-notch RSS reader (and beyond). Through Feedly I’ve annotated and added notes to thousands of articles over that time period.

When you get to this point, you do end up running into difficulty managing and searching. If I wanted to call up all my annotations on the war in Ukraine, the web app doesn’t have great search capabilities.

So all my annotations end up cast away. But Feedly does have an API. If you’re a Feedly Pro member, you can sign into Feedly and get a developer access token.

From there, you can use the access token to make JSON requests. In my case, Feedly has a GET method to fetch my annotations.

const apiCall = async (accessToken: string, path: string, method = 'GET', data?: any) => {
  try {
    const res = await requestUrl({
      url: `https://cloud.feedly.com/v3/${path}`,
      method,
      headers: {
        Authorization: `Bearer ${accessToken}`,
        Accept: 'application/json',
     },
     body: data ?? undefined
    })
    return res.json
  } catch (e) {
    console.debug(e)
    throw new Error(e)
  }
}

const a = await apiCall(accessToken, `annotations/journal?newerThan=${syncTime}&withEntries=true&count=100${continuation ? `&continuation=${continuation}` : ''}`)
  if (a.errorCode) {
     throw new Error(a)
  }
  return {
   entries: a.entries,
   continuation: a.continuation,
   count: a.entries.length
}

As you see, I’m looking to get all annotations after a certain point. I want withEntries to be true, as I’m looking to not just get the annotation but the metadata associated with the article. And while I can only fetch 100 at once, I can use the continuation token to fetch more in a loop.

What can I do all my annotations? Download them into Obsidian. I’ve already built a few Obsidian plugins, and this is yet another way to help me organize my life. Obsidian’s great search capabilities and backlinks feature will make it a lot easier to find my data.

I’ve setup a subdirectory, Feedly Annotations so that every annotated article will be represented as a file, with a filename sharing the title of the article. This will produce a lot of files, but at least they’ll be contained in my folder.

interface FeedlyAnnotatedEntry {
  annotation: {
    highlight?: {
      text: string
    }
    comment?: string
  }
  entry: {
    canonicalUrl: string
    id: string
    published?: number
    crawled: number
    author: string
    title: string
    origin?: {
      title: string
    }
  }
  created: number
}

I’ll get 100 entries at the time, and they’ll match the interface above.

function getInitialContent(entry: FeedlyAnnotatedEntry) {
  return `---${entry.entry.canonicalUrl ? `
url: ${entry.entry.canonicalUrl}` : ''}
feedlyUrl: https://feedly.com/i/entry/${entry.entry.id}
date: ${dateToJournal(new Date(entry.created))}
pubDate: ${dateToJournal(new Date(entry.entry.published ?? entry.entry.crawled))}
author: ${sanitizeFrontmatter(entry.entry.author)}${entry.entry.origin?.title ? `
publisher: ${sanitizeFrontmatter(entry.entry.origin.title)}` : ''}
---
`
}

Like other Obsidian files, I can take advantage of frontmatter to help organize data into separated structured key-value pairs. I can use the article metadata to add a number of interesting fields that I could search for in the future.

function getAppendContent(entry: FeedlyAnnotatedEntry) {
  if (entry.annotation.highlight) {
    return `

> ${entry.annotation.highlight.text.replace(/\n/g, `
>
> `)}`
  } else if (entry.annotation.comment) {
    return `
    
${entry.annotation.comment}`
  }
  console.warn('No append content for', entry.entry.title)
  return ''
}

const processAnnotations = async (entries: FeedlyAnnotatedEntry[]) => {
  for (const e of entries) {
    // Remove slashes.
    // File name cannot contain any of the following characters: * " \ / < > : | ?
    const sanitizedFileName = e.entry.title.replace(/[*"\/<>:|?]/g, '')
    const filename = `${sanitizedFileName}.md`
    const path = normalizePath(`${folderName}/${filename}`)
    let obsidianFile = this.app.vault.getFileByPath(path)
    if (!obsidianFile) {
       // Add the frontmatter
       obsidianFile = await this.app.vault.create(path, getInitialContent(e))
    }
    // Add the highlight or comment of this entry
    await this.app.vault.append(obsidianFile!, getAppendContent(e))
    entryCounter++
  }
}

For each entry returned from the API, I check if the file exists. If it doesn’t, I create a new one by providing the frontmatter. I append each highlight/comment to the file. This makes it possible for each article to have a complete set of my annotations without duplicates.

while (true) {
  try {
    const res = await getAnnotations(this.settings.accessToken, continuationToken, this.settings.lastSync)
    if (!res?.continuation) break // Exit out
    continuationToken = res.continuation
    await processAnnotations(res.entries)
    if (res.count < 100) {
       console.debug(`only got ${res.count} entries`)
       this.settings.continuationToken = undefined // Reset
       this.settings.lastSync = this.settings.continuationTime
       this.saveSettings(this.settings)
       new Notice('All Feedly annotations synced')
       break
    }
    this.settings.continuationToken = continuationToken
    // Save token for the future
    this.saveSettings(this.settings)
    console.debug(`>    at ${continuationToken} ...`)
   } catch (e) {
    if (e.message.includes('API rate limit')) {
       new Notice('The Feedly API rate limit has been reached, continue later')
    } else {
       new Notice(e.message)
       console.error(e)
    }
    break;
  }
}
new Notice(`Synced ${entryCounter} annotations`)

I run this loop continually, fetching all of the annotations or bust. There are a few ways I can bust. If I run out of annotations, then there’s nothing more to sync and I can stop.

As I’m using a development token, the API limit is not particularly high. If I tried to fetch all of the annotations, I’d hit an API rate limit and be out of API calls for a day. So I have added a bit of logic to save the continuation token between function calls in order to pick up the next day.

As shown in the screenshot above, the tool runs pretty well and is quite thorough. Now that I’ve got them all synced, I can very easily search for every article published by The Flux Review.

A caveat to the screenshot is that you may need to fix the titles afterwards. Since I’m using Dropbox to sync my Obsidan files, and Dropbox doesn’t allow emojis in the filename, this file title had to be fixed before it could sync.

When I ran this, and ran it a few times, I ended up downloading over 5000 files. I’ve been a very active user! And now that they’re so accessible, I hope that I can better take advantage of Feedly to make more connections and enrich my Obsidian vault further.

You can download Feedly Annotation Sync right now in the Obsidian Plugin list.

And the code for this plugin is publicly available on GitHub.

GitHub - Fleker/feedly-for-obsidian: Download my Feedly annotations