This is a great example that shows how poor most non-web debugging and logging. The ability to mix and match data types and presentation format is extremely useful. Almost everywhere else we just have printf. And that’s printf going to stdout — there isn’t a universally deployed format to even allow it to travel easily over the network and be viewed on one of many useful and advanced viewers.
I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it.
Damn, glad to see this, but they're also kinda encroaching on my territory! :P
I've been working on something like this that's language agnostic and works with not only tables but also trees, graphs, lists, hashmaps, etc., and animates the visuals as the data structures are modified in real time (and you can pause, playback, step, etc.): http://symbolflux.com/projects/avd
The api I was conceiving earlier was more complex, but if you read the copy there, you'll notice it's essentially the same now: `MObject.monitor(...)` or MTable.monitor() MStateMachine.monitor() etc.
Edit: really am happy to see some other projects like this in total, partly because I want to use 'em now if possible—but also because I've had a surprisingly hard time communicating to other developers why it might be desirable to do something like this in the first place, so I'll be very happy to have general awareness raised.
I read the first line of their docs as rather dry humor, "Displays tabular data as a table." —like yeah, of course you would want to see tabular data as a table, and hierarchical data as a tree, etc.!
At a quick glance it looks like it's targeted towards mutable data structures. Can it support (and would it make sense to support) immutable data? E.g. it would be nice to have a Haskell state monad that also visualizes its state.
You are correct, it's very much targeted toward mutable data structures. It'd be interested in looking more into the state monad and considering the possibilities, but I haven't yet.
Actually part of how I see the role of systems like this is part of the necessary conditions for more safely working with mutable data. I think mutable data is more risky, but also super easy to work with, so if we had ways of making it more reliable we should explore them. I envision languages where this sort of visualization is a core feature designed in tandem with the runtime etc.
Neat! Very interesting ideas!
The Vacuum package[1] seems like something you could look into if you want to explore Haskell.
Good luck with your project!
There are two clients in progress at the moment. By far the more complete one is for Java; the other is for javascript.
Since I've had such a difficult time communicating the potential benefit of the fully general piece of software, I've been exploring applications that I can build on top of it which do visualizations for more specific things (rather than leaving it to folks' imaginations to roam the wide possibilities of general data structure visualization :)).
As an example, my sister and I started building a 'code tutor' app in Electron, where you would be presented with problem sequences—but when you tested the code for your solution, you could watch what your algorithm actually did to the data (imagine a failed attempt to sort a list). This was the motivation for writing the javascript client. I integrated Lucidity's visuals by running it headless and streaming frames over TCP to the Electron app.
I think the next thing I'm gonna do though is just finish general object monitoring for the Java client, so that you could get something a little like the Chrome dev tools object printer in Java—but of course this one will allow you to watch algorithms performed on your objects :)
A problem I have with most log-like solutions is that them bombard you with so much info and you need parse them after the fact, yet none of the log visualization have a way for that (just ctrlf, with luck).
A simple improvenment? Put a search box in top of the displayer, and a way to make contextual search, (like :tree.at Code.py), and a way to save search patterns, so I can put the whole search syntaxt and just fill boxes (like :mySearch foo)
Hey mamcx—totally agree. Actually, I've got designs for something very much like what you suggest. I was actually thinking of designing a simple query language for it so that you could search for certain types of relationships etc. too.
Yes, sadly I rolled my own engine directly on top of Java/OpenGL, rather than using something pre-existing. (There were some decent reasons for it considering how specialized the visuals I'm working with are—but it was a big mistake over all. Should've done it in a more prototype-y fashion).
To code the animations I use something in the engine which resembles CSS transitions and can be attached to arbitrary Java objects, which I call 'property modulators'. So most of the animation you see comes from setting up one of those on one property or another.
Most of the rendering code would be a pretty direct port—I don't think I'm using anything from OpenGL that isn't supported by WebGL. That said, it is a desktop Java application and it would be non-trivial to port it to javascript. And that said, it probably would've been best to do it in javascript in the first place, but I did basically no web dev when I started, so... shrug
Only for those developers that insist in not using IDEs.
Common Lisp and Smalltalk environments have something like console.table since its early days.
Xerox even went to the trouble of adding similar capabilities to the Mesa/Cedar enviroment, because they thought they should cater to their Interlisp-D and Smalltalk developers.
Java and .NET even allow to provide metadata so that debuggers know what is the best way to render the data in debug view.
Visual Studio allows similar capabilities for C++ via data visualizers.
Now if you insist in vim and Emacs, there is printf.
I'm amazed sometimes at what people are able to accomplish banging rocks together and doing things the hard way.
Me, I'm lazy, so I lean on my tools to make my life easy. And there's some fantastic tooling out there; the biggest problem is usually discovering that a helpful feature exists and how to use it effectively.
Do you know how to record all backtraces of all threads and child processes with the VS debugger? Just curious. I did it lately with gdb [0], but with a bit guerrilla method. It was my try at localizing the origins of dreadlocks. I would appreciate pointers for something better for dreadlocks, but the original question still stands.
I understand that what I did would be a bit close to a poor man's reversible debugger.
Now if you insist in vim and Emacs, there is printf.
That's just plain wrong. GDB for example has had pretty printers for STL data structures for many years. Those operate on the underlying structure of the data types and you can write printers for your own data types pretty easily (gdb uses python for these).
Apart from that, there is structured hierarchical logging, which seems to be on the way of becoming an industry standard and is considered best practice in some ecosystems (golang comes to mind)
It has little to do with IDEs and everything to do with debuggers. When writing code in emacs or vim, as long as you compile with -g and run the binary with gen, you get the features you're talking about.
What's gen? It's not very googlable ('gen debugger' gives nothing relevant), and asking my package manager for a package providing 'gen' just gives some game.
Note that I'm not a GDB pro, I just recently started trying to use it more instead of printf, so there are probably lots of stuff I didn't show off. (For example, I wish I'd shown the `bt` command to print stack traces).
Another thing that's worth noting is that with GDB, if your program crashed and generated a coredump, you can use GDB to look at how everything was when it crashed, just like if you were debugging a running program. This is extremely useful on for example embedded systems where you don't have GDB installed, or when the crash is hard to reproduce.
I know that GDB is powerful and scriptable and very useful in environment where you don't have anything else. But I'm sorry to say, what you showed wasn't very discoverable and the UI was faaar from amazing.
I don't know if you've tried it, but whenever you have a bit of time try Visual Studio's (or even Visual Studio Code's) debugger. They're among the best visual debuggers out there. Or IntelliJ's.
You see directly all the variables as you step through code, you can see array contents directly, they're super discoverable, etc.
It's such a shame that there's no glory (or money) to be had in designing CLI/TUIs. I also get the impression that it would be really hard to change things, OSS maintainers have a reputation for being very stubborn about workflow changes (sometimes for good reasons, sometimes not) :(
GDB's main power is that it's scriptable. You can track down some pretty gnarly bugs with that. You don't necessarily need that capability for languages like JS, because there's far less danger of getting into serious trouble (stack corruption, etc.), so it's hard to appreciate. But yeah, discoverable it is not haha.
> Visual Studio allows similar capabilities for C++ via data visualizers.
No reason this couldn't be possible for Emacs, I guess nobody bothered. In particular, Common Lisp with SLIME already makes use of "presentations" in REPL, which nicely combine CL's print-object output with inspector/debugger capabilities.
Ultimately, no one limits you to printing out simple text. In various Lisp software I wrote, I ended up displaying profiling info inline in REPL, like this:
(EDIT: there were pretty Unicode characters in that output, but my browser / HN ate them. See https://github.com/tkych/cl-spark for how it should look like.)
(all it took was a simple macro on top of a sparkline lib). I dumped custom execution stats out-of-band into a HTML file that, with little bit of static JS, rendered explorable charts. I've sent data straight into new Emacs buffers for processing, with a help of this simple macro:
(can be trivially modified to make the new buffer automatically assume a given major-mode).
Point is, you don't have to limit yourself to simple, uninterpreted printfs. Write your own debugging tools! And while I'm not up to date with C#, most other languages and IDEs I've seen don't move beyond dumb printfs. Java definitely doesn't (hint: Grep Console is a very useful plugin to IntelliJ and Eclipse). And this Console.table() is just a visual gimmick. I'm actually disappointed that with all the capabilities for interactive HTML rendering, that's the best the Web standardized on so far.
When I think about Common Lisp debugging capabilities usually what comes to my mind is Allegro, Lisp Works, MCL, CCL, not SLIME in text mode single window.
Sure, but a lot of CL development these days is done with SBCL and SLIME - Allegro and LispWorks are expensive, and whatever cool stuff CCL has would require me to buy a Mac.
This would be fairly easy to build in Emacs for almost any language, but I can't see how this is so useful. Furthermore, if working with Org mode source blocks, it can render structured data as tables upon evaluation. Say you have this block:
(I don't use quote syntax to not confuse non-lispers)
Hitting C-c C-c on it produces the following block just below it:
#+RESULTS:
| col1 | col2 |
| 1 | 2 |
With Org source blocks you can have a literate program even using multiple languages and still communicating easily between source code blocks and both generated and static tables. It can easily be intermixed with formatted text and exported to a suite of various formats, including plain text source code (so-called "tangled source").
Edit: just for the sake of it, another example, this one in Python:
#+BEGIN_SRC python
return [[x for x in range(1,5)],
[chr(y) for y in range(ord('a'),ord('e'))]]
#+END_SRC
#+RESULTS:
| 1 | 2 | 3 | 4 |
| a | b | c | d |
The disadvantage of using box drawing characters is that they do not always align nicely - depending on the font, renderer, and couple other things. You can't go wrong with basic ASCII and a fixed-width display.
Beauty is in the eye of the beholder, true, but the majority of the beholders want a table to actually look like a table. If you give me pen and paper, I'm not going to draw weird half lines around the table, I'm going to draw the table first, line fully drawn everywhere and then add the content.
So much this! I've long wanted something where by I can chain, into a sort of tree, to more easily follow or hide large subcomponents and subservices. For example, say I make two calls in my service A to two other services B and C, and all report logs to a centralized location; that could get rendered as,
─ info: GET /foo/endpoint
├─ info: user <user ID> valid auth
├─ info: request to service B
│ ├─ debug: opening new connection
│ ├─ debug: success, result = ...
│ └─ info: request took 1 seconds
├─ info: request to service B
│ ├─ debug: opening new connection
│ ├─ debug: success, result = ...
│ └─ info: request took 1 seconds
├─ info: preparing result took 1 seconds
└─ info: http request took 3 seconds
with the ability to hide sections, perhaps grep for only certain messages (particularly if you keep the formatting and the message separate, this should be doable, I think), attach metadata to messages…
As it is, we have a fairly standard shove everything into syslog, and then pipe to a downstream logging system and a local file. But the downstream system is not very good at search (this is probably mostly our fault) and requires the message to be in JSON, so the stuff in the log file is _also_ JSON, b/c that's what syslog got. There are definitely better ways with our existing tools, but it sure makes one dream up what the perfect logging solution could look like.
> As it is, we have a fairly standard shove everything
Assuming "everything" is "static log statements" which is the first issue I have with this, a developer must have thought before deployment that this could be useful, which usually results in a lot of useless garbage and a lack of actually useful information. And little by little you build up actually technically useful data collection, and rather than being spread throughout the codebase all of the probes are centralised and readable.
I've been thinking about using dynamic instrumentation tools (bcc/dtrace) for that purpose instead, you know you need something when you actually do need it, at that point you can add it to the probes/instrumentation (which is external to the program and deployable separately), and all information would be collected in a structured form in a database you can interact with (probably not something relational).
If you're seriously interested in building this and would be open to employment for that purpose – please drop me a line (contact info in my profile). This absolutely deserves to exist and it would be a natural extension of what we're building at Scalyr. I'd love to talk.
Tracing calls across threads/servers is exactly the kind of stuff that should be easy and out of the box. If we had this built into a kind of universal logging protocol, you can imagine some viewers getting extremely sophisticated with network-level analysis, parsing & plotting times for distributions, anomaly detection, etc. Other viewers might be more geared towards mobile development, which might emphasize where network requests were generated in the code and its effect on the UI. Both should be possible from the same input data stream.
This is what I did my MSc thesis on. I should consider open sourcing the framework, even though it’s relatively “homework grade”
A couple of places to look though:
- opentracing.io
- Fonseca et al’s X-Trace work
I don’t have a link handy, and the visualizations haven’t aged well, but you can probably find a copy of my thesis or conference paper under Anthony Arkles in google scholar. I extended the X-Trace protocol a bit to make it easier to reassemble function calls that potentially had parallelism.
Because you see the black console almost everywhere. We debug tabular or structured data in visual debugger, console or specialized snapshot-view that is not hard to create. All depends on programmer’s will to make their life easier. Console.table() is just one of the simplest things that you could write for yourself in dynamic language, but the fact that most web “devs” cared to read the docs only after it hit top of HN makes it look amazing.
> shows how poor most non-web debugging and logging
Nonsense - web programming is inherently tied to the browser, which severely limits debugging and output options.
> The ability to mix and match data types and presentation format is extremely useful.
That's a common feature of dynamic type languages.
> Almost everywhere else we just have printf.
Most languages have logging libraries, and many have them built into the stdlib or runtime. Popular languages usually have many logging libraries to choose from.
> that’s printf going to stdout
Of course - printf(3) by definition "shall place output on the standard output stream stdout"[1]. If you want stderr or any other output stream, use fprintf(3).
#include <stdio.h>
int main() {
fprintf(stderr, "Error: %s\n", "LP0 on fire");
return 0;
}
Other languages usually handle that for you as part of the logging library.
> there isn’t a universally deployed format
Sure there is: "text"[2].
> to even allow it to travel easily over the network
Copying files to a remote host is easy with scp. There are numerous ways to send data over a network. Web software is often more work as you have to move your debugging data/logs out of the browser first or use a specialized tool that does that for you. Sending program output realtime to a remote host only requires appending "| ssh ${host} \"${remote_cmd}\"" to your command.
> advanced viewers
That's great if you have specific needs and a viewer for your complex data structures, but the only universal solution is text. Fortunately, there are many tools available to handle structured data as text.
> I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it.
I think you might want to explore the work programmers have done in other languages.
> Of course - printf(3) by definition "shall place output on the standard output stream stdout"[1]. If you want stderr or any other output stream, use fprintf(3).
I reject your C-centric world for one of my own making. ;)
# perldoc -f printf | head -n3
printf FILEHANDLE FORMAT, LIST
printf FILEHANDLE
printf FORMAT, LIST
# perl -e 'printf STDERR "%0.2f\n", 2;' >/dev/null
2.00
You're totally missing my point(s). Of course web programming is tied to the browser, but why does that mean debugging via logs has to be tied to whatever particular console is attached to where a program is running? Why can't I have a console.table() equivalent in Python, and Ruby, and C, and Go, and Java that all can render via a common protocol to any viewer that can handle it?
By printf I didn't _literally_ mean printf -- I meant any logging type equivalent whether that IS printf or a built-in logging system. The point is there's no inter-op between them with any rich detail, which prohibits universal log viewers.
When I say travel over the network, I don't mean scp. I mean allow my iOS/Raspberry Pi/local Erlang server device to stream in real-time it's logs to any viewer that I want, whether that's on the same device or somewhere else in the world. That's the kind of portability required when you can't guarantee that complex display can be part of the same system that generated it -- that's a unique quality of web browsers. IT'S ABOUT THE PROTOCOL, NOT THE MECHANISM.
Text is great, and most logs will be in text. But I also sometimes want to inspect data, or expand JSON, or view an image, etc etc. But let's even stick with text -- a good viewer would let me set up filters on a host of structured qualities -- filenames, lines, log levels, keywords, etc. It would output in a way that allows me to easily view such info in a UI that supports infinite scroll back, easy copy/paste, elegantly formats so different log instances are clearly separate from each other, adds color to enhance readability, allows me to hide/collapse sections or duplicate entries, provides columns for the structured logging fields like timestamps, allow interactive sorting on these columns like Excel, etc etc all without losing my data.
Apple's Console.app is an elementary example of what I'm suggesting I'd like to see. Except I don't want it tied to the syslogs of an Apple device. I want every language/platform to be able to output to it in real-time. And for it's capabilities to be as rich or richer than what we have in the browser. We already have universal text editors like VIM/Emacs -- why can't we have universal log viewers?
> Text is great, and most logs will be in text. But I also sometimes want to inspect data, or expand JSON, or view an image, etc etc. But let's even stick with text -- a good viewer would let me set up filters on a host of structured qualities -- filenames, lines, log levels, keywords, etc.
So, it sounds like you want a debugging library you can include in the language you are using to output structured debug lines and a client side that goes along with it to represent the usefully (or just use text+JSON and a JSON aware client), and to just stick it on top of syslog. I think that pretty much covers what you're asking for.
>So, it sounds like you want a debugging library you can include in the language you are using to output structured debug lines and a client side that goes along with it to represent the usefully (or just use text+JSON and a JSON aware client), and to just stick it on top of syslog. I think that pretty much covers what you're asking for
Except the most important thing: that ability to be included as standard, readily available, and well supported in languages.
Well, that's why I specified it as a library. A C compatible Lin is likely easiest for most languages to trivially support through an FFI, and it all it's doing is outputting structured text based on the input, that's trivial to hook up. It's not like it can be too language specific because if it's supposed to work in a bunch of languages, it has to be generic enough to handle them.
Although in the end you probably aren't getting much out of it, as a dynamic language is fairly easy to just pump stuff through a JSON converter, and static/strongly typed ones probably require enough boilerplate that the added benefit is slight.
"printf" will always have it's place, because when things are hosed, there's a good chance you can't push stuff to network, but things have to be really hosed to stop you pushing some chars to a UART.
> I mean allow my iOS/Raspberry Pi/local Erlang server device to stream in real-time it's logs to any viewer that I want, whether that's on the same device or somewhere else in the world.
> But let's even stick with text -- a good viewer would let me set up filters on a host of structured qualities -- filenames, lines, log levels, keywords, etc. It would output in a way that allows me to easily view such info in a UI that supports infinite scroll back, easy copy/paste, elegantly formats so different log instances are clearly separate from each other, adds color to enhance readability, allows me to hide/collapse sections or duplicate entries, provides columns for the structured logging fields like timestamps, allow interactive sorting on these columns like Excel, etc etc all without losing my data.
"Why can't I have a console.table() equivalent in Python"
I get a console.table() equivalent in Python by using Jupyter Notebook, which is a web-based REPL shell, and pandas, which is a library for working with tabular data. Pandas displays HTML tables when used with Jupyter.
This response managed to say very little and just give excuses for the current state of things.
>Nonsense - web programming is inherently tied to the browser, which severely limits debugging and output options.
Web programming includes cli and server side tools (Node for one, JS testing tools, etc), has several first class logging SaaS and locally installed options, and even has stuff like the developer tools protocol, which allows one to debug Chrome running code in VS Code for example...
>That's a common feature of dynamic type languages.
That's merely an "ability", it's by no means a "common feature" in dynamic languages to have different output options in the example of e.g. console.table() (which itself is limited anyway).
>Copying files to a remote host is easy with scp.
That manual busywork is not what the parent asks for.
>That's great if you have specific needs and a viewer for your complex data structures, but the only universal solution is text.
You've just restated the problem -- only with a positive spin.
A proper debugger like the one in VS and general structured logging would like to challenge that statement. I'll take those over what you have in the web space any day.
Chrome developer tools will do all of these except
> * Record execution history and state so you can debug after the fact for hard non reproducible bugs.
^ Which would frankly be awesome. Various libraries provide their own userland support for this but it's all fragmented with hugely varying levels of quality and flexibility, but out of the box support would make me so happy (unless this already exists in which case, someone please enlighten me!).
A couple years back at university, I created DoodleDebug[1,2], tackling some of your mentioned issues. It can be thought of as a graphical version of System.out.println(). There is a similar output format customization mechanism as toString(), but you can also customize the rendering of arbitrary types you don't control the source code of.
DoodleDebug's output is HTML-based and interactive with semantic zoom. That means you can click parts of printed objects to inspect them.
However, I didn't work on it in years due to lack of time, and even the linked description page is a bit outdated (it's not an Eclipse plugin anymore).
One of the under-shared features of Rust that I adore is #[derive(Debug)] -- having object output like a dynamic language on low-level, statically typed struct definitions is AWESOME.
Huge boon to productivity, especially when leveraged in tests.
I wish PowerShell and/or some of its ideas had a bit more mindshare outside of the MS ecosystem.
Other folks have talked about this at length (e.g. https://mkremins.github.io/blog/unix-not-acceptable-unix/), but the fact that there is no standard way to pass anything other than strings of plain text between UNIX shell commands is incredibly limiting.
Heck even if we could pass a flag (say, -j) to something like ls and have it spit out JSON instead of plain text would be an upgrade.
If you are debugging a program inserting printf or other debug prints in the code you are doing it wrong, for debugging you should use a debugger.
It's slow and inefficient, you add the debug print, then you need to recompile the program, maybe you need another debug print, recompile the program, and so on, and then you must remember to delete all the debug prints that you added! And let's hope that your debug print didn't add some strange side effect (they might have!).
With a modern debugger you could print everything you want, at every memory location, you could set watchpoints, isn't it better than printf ? Also where you printf on an embedded system...
Sometimes you just can't do that, without completely breaking everything while the program is halted in the debugger. Particularly real-time, multithreaded applications.
Not necessarily. It's about timescales and other environmental factors. Pausing in a debugger is measured in seconds. Your network-using application won't notice extra half ms for a print statement, but might change its behaviour if you pause it for 5 seconds, as requests will timeout. A video game running in fullscreen won't mind printing to console/file in the background, but may not handle well being interrupted in the middle of rendering and having you switch focus to the debugger (less of a problem these days, but I remember this being a pain ~10 years ago).
Sometimes you may not want to pause execution. Adding a print statement is often faster than starting a debugging session - especially in languages like Common Lisp, where you can simply recompile a single function of a running program. Another benefit of print statements is that they're append-only, not self-overwriting - that is, I can stick one inside a loop and quickly collect a bunch of values to review, or even copy over to another tool for analysis.
A modern debugger is a powerful tool (especially if the language you're using isn't itself very powerful). It's worth knowing your way around it. But that doesn't mean you shouldn't be using simple print statements whenever that's faster or more useful.
If you use something like Symbolics Genera or CLIM, the printed output is actually still connected to the data. Thus you can use relatively primitive PRINT functions and still be able to drill into the objects later.
I use often in Lisp the INSPECT function with a GUI inspector. Each call to INSPECT puts the data into the inspector. For example (inspect (list :loop-i i :value n)) puts the list with the data into the inspector. The inspector has a history, which then allows me to see the various items. Thus this is similar to a print statement, but records the actual objects into a separate tool.
For the sake of completeness, I'll add that the output-connected-to-data works to some extent with Emacs & SLIME as well. These are so-called "presentations" - if you print an object, SLIME will connect the printed text to actual Lisp object, which allows you to inspect it later, or even copy and paste within the REPL - as long as you copy and paste the whole "presentation text", the underlying association will remain.
Sometimes it's just easier. Especially if you have a nice incremental build system or an interpreter to which you can just send expressions and definitions and see the results.
Edit: also, you don't need to remember to weed out debug prints, you can, in almost all languages, define a function or a macro to do that which can be undefined or an ignore function if not built in debug mode. With cpp or Lisp, that'd be a compile time decision, and I suppose many languages could optimise out a function that is defined to just ignore it's arguments and do nothing.
Clojure's not often mentioned clojure.inspector namespace contains two functions: inspect-table and inspect-tree. They roughly approximate Console.table() and Console.dir(). I tend to get more mileage out of inspect-tree, but even so I find it limiting. I dream for a graphical object diffs and a version that when passed an atom or ref, auto-updates when it changes.
On the one hand, I like it. It's good this function is there.
On the other hand, I'm disappointed. With all the power the web browser tools have - basically, arbitrary HTML rendering in console - this is the best we get? A simple, dumb table? There are so many to be explored here!
> I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it.
A lot of programmers aren't aware that yes, they can and should modify their environment to suit their needs. Yes, they are allowed and should write their own debugging tools. Those tools don't have to be complicated. You can start with an ASCII-based console.table() equivalent. Or a script that parses the output and feeds it to some half-baked D3-based visualization. Or even gnuplot. Or graphviz. Plenty of possibilities there, and yet most devs I know never even think of them.
I want a thing that can plot a numerical array as a line graph of data points. A histogram and a scatter plot would be nice, too.
And I don't mean for computational/data science exploring, I've got numpy/matplotlib for that. But really as a debugging tool in the browser's dev console. It's not uncommon that I have an array of numbers (or something that looks like one), a few hundred or more. Finding the bug or unexpected behaviour in my program is often about finding the numbers in such a list that "stand out" in some way. Now I could specify and filter those elements that "stand out", but I could also plot a graph and grasp its behaviour in a single glance. Just dumping the numbers to the console and hunting for aberrant behaviour likely takes me out of my flow, where a graph would not.
Yup. This is something I'd love to have too - in particular, point/line plots and scatterplots at resolution greater than possible in text-only terminals. I already render text-based histograms when I need.
Come to think of it, it shouldn't be hard to extend Emacs to eat specially prefixed output (e.g. #<scatterplot: (1 2) (3 4)...>), feed the data to external program and place a rendered chart inline. I might pick it up as a small project. Thanks!
> there isn’t a universally deployed format to even allow it to travel easily over the network and be viewed on one of many useful and advanced viewers.
My absolute favorite web-console feature (in Chrome and Firefox at least) is the ability to go back in the REPL history to edit a previous function definition. A standard REPL just gives you back one line at a time, so if you want to make a change to line 3 of a 4 line function, then you have to type:
UP UP UP UP ENTER UP UP UP ENTER UP UP
before you can edit that line. But the web consoles allow you to type UP just once, and then move your cursor around the entire function definition. I wish all REPLs had this feature.
enables searching by the current line's prefix in all programs that use readline (a lot of languages use readline for their REPL). That is, if you type "cp<UP>" you immediately get the previous command that started with "cp", regardless of how far back it is in your history.
This is a useful trick, but I don't see how it addresses the issue they talked about. As I understand it, they were saying that Firefox and Chrome's Javascript REPL's scroll up through expressions rather than lines. If you say "const f = () => {" and press enter, it doesn't submit it until you close that brace, and when you use the up arrow to scroll back to it later, you get the whole thing (newlines preserved) instead of single lines of it.
This is trivial in the right environments. Not shockingly, in emacs I get that.
It actually takes a little getting used to at first, that up literally just moves the cursor up into the output of the previous command. Nowdays, it is severely restricting to have the same commands move the cursor that change the current command. (That is, left/right move cursor, up/down do a ton more in most terminals. That is not the case in emacs shells.)
Really, though, the repl is a fun stepping stone to just sending function definitions from the file you are working on into an environment. https://github.com/skeeto/skewer-mode is a fun video showing the idea for javascript, amusingly.
That said, there was a fun video a while back of the guy that did Minecraft live coding a video game. Basically, the game would be running, and he could make a change and it would live update, just like the javascript video above. But it was in java.
This is harder in non-managed runtimes, I believe. But not impossible.
Good point! The first time I used firebug, probably 3 years before I used Chrome, it blew my mind and immediately made clear how much faster it was going to make styling pages, debugging errors, and just testing JavaScript expressions.
While I agree it's nice they can take it, I don't think their viewer is particularly readable or advanced. Apple's very basic Console.app is a much stronger example, and it's super simple in capabilities.
We are too hell-bent on that unix philosophy that everything is a text, stdin, stderr and pipes are enough. It is not. We should have ways to log objects. We should have ways to pass those objects around and serialise, deserialise and inspect them.
It's not just text. Grep is not enough for investigation.
I think the closest thing for this would be something like SpyJS, which is available for both NodeJS and browser JavaScript in JetBrains products such as Idea and Webstorm.
You can see previous executions of the code you're interested in, with a highlight of the paths followed through the logic and snapshots of the variables as they were at the time.
Well, since that time when chrome was unable to find the javascript file of the page I was looking at, forcing me to go back to internet explorer, I disagree.
Also is it possible to edit the javascript code without reloading the page? I know how to do it with java+eclipse, but not in the browser.
If you use Chrome DevTools, there are a few resources that I think are really useful for getting the most out of them and making the workflow a bit easier.
I was just playing around in node a few days ago, and had to mock some console statements. I stumbled upon this exact API and played with the output bit. It's a useful feature I never knew about in my years of doing javascript and it's kind of surreal to see it on the top of HN just a few days later!
I made something similar as a fun little project https://github.com/mrmagooey/taboo, does the same print function but also does some basic querying, deletes and updates, left and inner joins. I had no idea that the inbuilt version existed though.
Recently I've been working on kind of the converse of this problem with JSON and spreadsheets, and I'll briefly describe it here (and I'll be glad to share the code), in the hopes of getting some feedback and criticism:
How can you conveniently and compactly represent, view and edit JSON in spreadsheets, using the grid instead of so much
punctuation?
The goal is to be able to easily edit JSON data in any spreadsheet, copy and paste grids of JSON around as TSV files (the format that Google Sheets puts on your clipboard), and efficiently import and export those spreadsheets as JSON.
It's especially powerful with Google Sheets, since you can run JavaScript code in them to import, export and validate JSON, and other apps can easily retrieve those spreadsheets as TSV files, which are super-easy to parse into 2D arrays of strings to convert to JSON.
This is a new reimplementation of some old ideas I've been happily using for years with Python and CSV files, plus some new ideas. But now it's general purpose, JSON oriented, written in JavaScript, integrated with Google Sheets, and I'm releasing it as open source once I get it into working shape.
Here is an example spreadsheet, which includes several sheets of different structures, and a script to convert the spreadsheet to JSON.
The first sheet "world" is the top level object, which configures a Unity3D app scripted in JavaScript (that's UnityJS, another story, but just think of this JSON as an arbitrary example). So the world is a big fancy JSON structure used by JavaScript code to create and configure a bunch of prefabs.
As you can obviously see, you just write your JSON expressions as an indented outline with type names before the values, putting keys, types and values in separate indented cells.
object
tileRows number 1
tileColumns number 2
materialTiling object
x number 4
y number 4
materialOffset object
x number 0.5
y number 0.5
...
You can use the "sheet" pseudo-type to make references to objects, arrays, etc, defined in other named sheets of the same spreadsheet:
And the prefabMap sheet is just a map of different types of content, directories and file names. You can represent any kind of JSON structures in the spreadsheets, and it's much easier to edit than raw JSON text.
Also, there are some convenient ways of compactly representing repetitive structured data.
One common type is a 2d array of elements all the same type, which you can make with the "grid" pseudo-type:
That's a 10x10 grid of strings. You can also make grids of other types that will fit into one cell. For example, you could make a grid of strings or numbers or booleans, but not arrays or objects, since they don't fit in a single cell. But you could make a grid of the "sheet" pseudo-type, with a grid of sheet names, each sheet each containing any type or pseudo-type of object.
It also supports a "json" pseudo type that lets you put an arbitrary JSON expression into one cell.
Notice that the values themselves may be calculated by spreadsheet formula, which call any function or refer to any cells in any other sheet. In this example, each tile name is calculated by randomly choosing from a range pointing to all the hex tile names in the prefabMap:
You can write comments and intermediate values and formulas off to the right of the JSON data, or in other spreadsheets, which the JSON data may ignore or depend on.
The spreadsheet gives you enormous amounts of power to dynamically process the resulting JSON! So you can publish spreadsheets connected to live data, and import them as JSON.
Some people laugh at me when I say I prefer spreadsheets to outliners or even text editors. I'd love it if there were a decent tree structures JSON editor as powerful and ubiquitous as Google Sheets, but I've never heard of one. Tell me if you have, please!
Userland Frontier (later Radio Userland) came close to that ideal, predating and then adapting to both XML and JSON, but it never quite hit the mark of what you can easily do with a spreadsheet.
Another great way of compactly representing repetitive JSON structures in a spreadsheet is to use a table whose headers specify the structure, key names and types of the JSON arrays and objects.
This is essentially what Console.table does with an array of objects, but with rows and columns exchanged, and supporting arbitrarily nested arrays and objects.
The "bowConfigs_outline" sheet has a conventional outline formatted array of two JSON objects (rainbow bow configurations) that have the same structure. It is 175 rows, and 9 columns (1575 cells, many empty, with many repeated types and key names).
array
object
bowStart number 0
bowEnd number 0.8
bowHeight number 30
startWidth number 1
endWidth number 0
widthMultiplier number 5
fromLocalOffset object
x number 6
toLocalOffset object
x number -6
...
The "bowConfigs_table_compact" sheet shows the compact table representation of that same structure, with headers that implicitly describe the shape of the structure so there is one value per column, with no rows or columns of wasted space. It is only 4 rows (a row with the "table" type, a row with the headers, plus one row for each structure), and 58 columns (232 cells, few empty, with no unnecessarily repeated types or key names).
Lining all the values of the same name and type up in vertical columns makes it really easy to view and edit the values, and apply spreadsheet formula to dynamically calculate them! It would be much harder to track down and apply a formula to each "bowEnd" property in the outline format.
(Shown here as comma separated values since the headers contain spaces and that would be confusing.)
The top row is a series of columns of tokens, including [ and ] for arrays, { and } for objects, and keys and types for the object and array elements. It's like a simple horizontal schema that specifies the JSON structure, key names, types and columns, so that nothing needs to be repeated, and no space is wasted.
table
,{ bowStart number,bowEnd number,bowHeight number,startWidth number,endWidth number,widthMultiplier number,fromLocalOffset { x number },toLocalOffset { x number },lineRenderer/startColor { r number,g number,b number },lineRenderer/endColor { r number,g number,b number },lineRenderer/alignment string,lineRenderer/widthCurve { animationCurveType string,keys [ { time number,value number }, ...
,0,0.8,30,1,0,5,6,-6,0.8782983063,0.7984165253,0.0370873959,0.7169641118,0.7843719274,0.3921475355,View,Keys,0,1,0.25,0.2,0.5,0.5,0.75,1,1,0,Blend,0,1,0.25,0.5,0.5,1,0.75,0.5,1,1,0,0.9368487271,0.6433703118,0.198860128,0.25,0.4861432977,0.5704963395,0.6107422953,0.5,0.9640410996,0.08846161285,0.05927839517,0.75,0.1199717053,0.2262674866,0.7876422776,1,0.6955264667,0.01858220912,0.7418451801
I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it.