When I was working in construction (electrical) we would “predrill” our grounding rods about 75% the length using this method. These are normally 1/2-inch diameter, 8ft long rods that must be driven via rotary hammer but the bedrock being so close to the surface and Sacramento valley riverbed clay density was just too much sometimes for the hammer.
Perhaps as a profit center to the tea industry. Most tea bags contain the grades of tea that can't be sold in the more expensive loose leaf form. Basically leaf fragments and dust. That material would otherwise be a waste product.
I’ve been using it to great effect for the last year or so. It really is a breath of fresh air. In my case, I’m using a Django backend, but there are lots of successful folks using other backend stacks as well so choose what you know best.
The book is a really quick and easy read. It’s pretty eye opening how productive, and effective, htmx can be with so little magic.
Ideally you’d used Parquet or ORC if querying from Athena/Trino/Presto. Since they are columnar formats you will enjoy considerably faster queues and lower query costs for most query patterns other than “select *…” since the query engine can just retrieve the columns needed for your query instead of the entire row.
Ideally you will also partition the data, so your queries can use partitions. Athena charges $5 per terabyte of data scanned, so it's important to get this right.
I was experimenting with the Warp terminal for the first time last week. It includes a similar feature. I was pretty impressed with how well it worked.
I just tried polars for this first time this week. I ported a data pipeline from pandas and I was blown away by the performance yield. Function went from a 60 min runtime with pandas to ~1:30 in polars!
I’ve been using pandas for years and had no issues picking up the syntax. Can’t recommend giving it a try enough.
By any chance were you iterating over your pandas dataframe or using .apply? I’d be surprised by any properly formatted (i.e. vectorized) pandas operation that takes that long for data that fits in memory
I'm not saying that polars isn't faster. In fact in my other comment here I mention that polars is much better than pandas at what polars does (it's not a drop in replacement). I'm just saying that most of the times (not always, and in fact in those cases we've used polars to speed it up) that I've seen painfully slow pandas operations has been due to poorly formatted pandas code.
I recently inherited a codebase that uses Pony ORM[0]. Might be worth taking a look if you want your interactions to look as similar to vanilla python expressions as possible.
One of the features I like in ES that I haven’t seen in alternatives is “Percolate queries” (queries where you feed the service a document and it returns a list of queries that you’ve indexed that would match that document - basically inverting the whole process).
Does anyone know of any alternatives that support this use case?
[0] https://duckdb.org/docs/archive/0.8.1/api/python/relational_... [1] https://github.com/duckdb/duckdb/pull/8083