Author of Wordcab-Transcribe here. We use faster-whisper + NeMo for diarization, if you want to take a look.

Powered by BART and trained on my own dataset of a few thousand summaries. Let me know how it works for you :)

login: test

password: test123

login: test pass: test123

A few years back I discovered Python, and very quickly after NLP. To a writer with a love for sci-fi/tech, I was enamored, and spent ungodly hours on my employer's GCC-hosted Jupyter Notebooks, coming up with all sorts of impractical experiments with Spacy, Facebook's Starspace, Gensim, and the like.

For one, I needed a lot of training data. I'd go crawl thousands of pages of text from news sites, using Scrapy and storing data directly on the server. For text extraction and boilerplate removal, I used newspaper3k, and eventually a custom extractor that used a random forest model to select proper element "candidates".

I wanted a simpler way to aggregate text for a dataset, query it, create subsets based on keywords, and so on. The paid options out there - Diffbot, Aylien, Ujeebu, Scrapinghub's news API, etc. weren't exactly what I was looking for.

After learning the minimum amount of JS required, I built a shitty local app where you could paste a bunch of URLs and get back a JSON with the extracted text. I posted it up here, on HN, and there were a few hundred visits, absolutely demolishing the $5 DO instance. I figured others might want something like this.

So I built extractorapi.com - a text extraction API and UI that revolves around the idea of "Jobs". For example, let's say you gathered a list of URLs from the NY Times, or The Economist, or Bloomberg. You then provide that list of URLs to a job called "my_articles". For example:

api_key = "YOUR_API_KEY" endpoint = "https://extractorapi.com/api/v1/jobs"

headers = { "Authorization": f"Bearer {api_key}" } data = { "job_name": "my_job", "url_list": [ "example.com/article1", "example.com/article2", ... ] }

r = requests.post(endpoint, headers=headers, data=data)

This job will then process your input URLs server-side, and once complete, you can query the extracted text or title within the job. All jobs and extracted text are saved on your account - you can use the API or the web app to explore the jobs you started programmatically, download them in .csv or .json formats, and check their progress.

I go into more detail in this Medium piece, "Creating an Automated Text Extraction Workflow": https://medium.com/@aleks_82234/creating-an-automated-text-e...

I get that it's hard to market to devs like myself (why buy when you can build?), so I'm looking for any feedback/criticism/suggestions on your experience with Extractor API. As Faulkner points out, you must kill your darlings - let me know if all this shit doesn't make any sense, or if it's actually helpful. Or somewhere in-between.