By the same token, in Lisp, Sexprs aren't a serialization format. They're a notation for the linked cons cells that Lisp data is made of. In Lisp, that Sexpr will be crawled for data, or maybe even executed.

So while in Python, both may seem to be serialization formats, they aren't.

Either way, if the application programmer has any sense, they'll abstract away the format of their data. In a lisp app, you won't be cdring down a sexpr, you'll be calling a function to grab the necessary data for you, usually from a set of functions that abstract away the underlying sexpr implementation, and treat whatever it is as a separate datatype.

Of course, the sexpr might have been fed to an object constructor. Heck, it might be an object constructor, or a struct constructor. All of those types typically provide O(1) access, and autogenerated access functions, so it's the same story.

> In Lisp, that Sexpr will be crawled for data, or maybe even executed.

A s-expression cannot be executed; it's just text.

The object which it denotes can be walked or executed via eval.

Before that happens, the s-expression must be converted to that object.

In other words, deserialized by the reader.

This feels like you're hairsplitting to no obvious benefit other than increasing confusion. I could as easily say "mathematical notation isn't math it's just text, you can't evaluate '1 + 2' without a human being because otherwise those are just marks on the page otherwise." This is true(ish) (with the correct escaping), but it's difficult for me to see how it's relevant to the discussion? We could imagine the situation where I've created the Texpr, which has slightly different notation but the same properties. I don't know that we would necessarily classify it differently or treat it differently.

This leads to the alternate conclusion that maybe the s-expression is the underlying set of objects in the interpreter/compiler. (S-expressions are a special type of linked list, in that case). This rings true(er) to me, because of the way that we talk about s-expression manipulation in lisps. We most certainly are not using string operations to generate them. In which case the thing with the many parenthesis is merely standard lisp syntax, not s-expressions. This is further justified by the existence of different syntax in Dylan or Clojure, and availability of reader macro manipulation as its own entity.

Tldr; it most certainly is not text! Nor can it be executed.

That is correct. It's just text which talks about concepts that don't have text, like transcendental numbers, infinities, infinitesimals and so on.

There are functions in mathematics that can't be written down in symbols at all, like the integrals of certain functions (which themselves can be written down).

Math text has some useful properties in that certain transformations you can think of as typographical (manipulations of the text) actually preserve semantic properties in a useful way. So for instance addition commutes, semantically; and in the text, this lets us swap the left piece of text for the right one, around the plus sign.

There can be a very close correspondence between typography and semantics (like in Douglas Hofstadter's "TNT": typographical number theory, which he uses to explain Gödel's incompleteness theorem).

> This leads to the alternate conclusion that maybe the s-expression is the underlying set of objects in the interpreter/compiler.

I assure you that it isn't; not in any main-stream Lisp interpreter or compiler.

(Where by "main-stream Lisp interpreter or compiler", I intend to rule out cute hacks like this:

https://news.ycombinator.com/item?id=7956246 )

The problem that arises in Python and other dynamically-typed languages is that there exists a default deserialization that is so good it very strongly tempts the programmer to use that exclusively. However, as good as the default serialization may be, it's also quite dangerous, for the reasons you mention and more. In strongly-typed languages there's a stronger focus on parsing the JSON instead, which has the advantage of producing objects with stronger guarantees (which isn't quite the same as pointing out there's stronger typing here, you could theoretically get the same guarantees in Python with some sort of JSON schema library or something), but has the disadvantage of generally being more challenging, since it's hard to beat the conciseness of "json.loads(s)" in Python. There generally is a default serialization in strongly-typed languages, but it's far more likely to become inconvenient if you need anything beyond simple numbers and strings, and people generally learn to prefer true deserialization in my experience, unless they, alas, start their program out from day 1 inputting and outputting JSON and accidentally structure their entire program around the default JSON structures and end up with the exact same problems as you'd get in Python. But as long as JSON isn't the very first thing to go in you're generally in better shape.

(I have witnessed a Java program primarily written as a map of string to map of string to map of string to string. It was unsalvagable. And for all the cynicism I may occasionally muster, I don't say that often, because refactoring can be pretty powerful in Java, but this was beyond help. It actually had no JSON in site, but the same fundamental forces were in play.)

Personally, despite preferring the more strongly typed approach in most ways, I must confess that when I'm working in Perl I am generally unable to resist the temptation to just JSON::XS::decode_json, and cover over the differences with unit testing rather than dealing with "true" deserialization. I make myself feel better by also telling myself that if I do anything else, I will confuse my fellow programmers who don't generally expect to see fancy deserialization routines when dealing with JSON, which is true enough, but in my heart I still know guilt.

I think you mean _statically_ typed languages here. Python is a strongly typed language.

So I don't use the useless definition. By any useful definition of strong vs. weak typing, that is, one that actually creates two or more non-empty, non-trivial sets of members in the universe of discourse, Python is a weakly-typed language.

Furthermore, one of the most popular languages in use today is weakly typed: JavaScript!

Note that I do not endorse using "strongly-typed" to mean this definition, or any other. There are no useful definitions of this phrase, don't use it, except when correcting people.

This works well in Lisp, where the line where objects end, and structs, lists and functions begin is hazy at best. Besides with a bit of wrangling, you could probably just pass your validated serialized data to the object constructor as the arguments. Or you could just write a struct, which is simpler than an object, provides O(1) access, and you can still probably easily pass your datastructure, or something close, into the constructor as the arglist.

Whilst that's more possible with protocol buffers or pickling (or whatever your language calls it), I can't think of any languages offhand which can round-trip any data. It's generally not possible to serialize objects denoting external resources - such as open file handles or network sockets. It's also often not possible to serialize closures, weak references (without dereferencing them), and not necessarily possible to serialize self-referencing objects: e.g. a list which contains itself - pickle can handle it, but I don't believe protocol buffers can do it.

fds and sockets CAN, IIRC, to some degree, be sent to other processes on the same machine, but it's fairly limited.

As for serializing closures, CHICKEN Scheme's s11n egg is the most prominent, although not the only, example. It's fairly limited, once again, to avoid sending the forest with the bannana, as Joe Armstrong would put it.

This has nothing to do with our discussion, I just thought it was cool.

http://json.org/

It's nothing about pressing into service. This is the authoritative source.

http://www.korea-dpr.com/

It's democratic, and for the people. It says nothing about being a totalitarian dictatorship.

Not everything is/does what it says on the tin. You don't have to agree with official or otherwise authoritative sources without question.

(not that I wish in any way to compare Mr Crockford or whoever runs json.org with DPRK or its leadership - I'm just using a deliberatly extreme example to highlight that what is written may not be what is, at least not from absolutely everyone's point of view)

I generally do, thanks, though I'm not any sort of genius by any measure.

> because in context, this is far too inapplicable to the discussed reality of JSON to be an actual point.

You seem to be missing a bit of an intentional context switch. The comment was more about the logic of the GP's response to "<something> isn't really X" which was "yes <something> is X, it says so on <something>'s home page", than it was about <something> or X in particular.

So it was relevant in the context of the discussion and the facts being used for reference but not, as you call out, in the context of the subject of the discussion (hence my somewhat defensive clarification of intent in the last sentence)

For one, serialization usually handles binary data as well. And types, lots of types. Serialization includes class definitions, etc, which are used to create a (in this case) python object. JSON is literally javascript object notation, and has nothing to do with pickling python.

Serialization doesn't have to handle a particular type system in its entirety to be serialization.

A serialization scheme can dictate its own type system; which can be smaller than that of the programming languages which support that serialization scheme.

JSON has a simple type system; it serializes that system.

(Might you be confusing serialization for other concepts like object store databases, or image saving?)

The point is, the syntax behind JSON was originally designed for a specific language, as a textual representation of that language's objects. It just happened to make a convenient serialization format.

JSON isn't a strict subset of JS. JSON strings can contain literal line terminators. JS strings cannot.

It's a bit more tricky in JS, both client-side and node. You can't work with the json string there, but after that you work directly with the json object. They're not OOP languages, really. I wouldn't want to work with too much untyped / unstructured json in back-end land myself to be fair.

To give you a flavour of the kind of poblem: In C# (or rather .net) json.net reads JSON and calls setters from the target class.

That means the setters have to be public, and you don't know what order they will be called in, and you have no real signal about when it is all done. The constructor is no longer enough to guarantee the object's invariants are met.

Most awkward.

Perhaps I should have done it for the easy cases as well (where the business objects are struct-like enough that it doesn't matter) and just lived with the boilerplate.

But I see little advantage in this over just having a dictionary that I can inspect to initialise my real business object. True that is not strongly-typed, but the stage between the message-object and the business object can have validation errors anyway, so why not treat typechecking as part of that?

Is it really that hard in c#? It is not something I ever think about in Java.

You do have to use thin constructors, but in JSON.NET there's a way to call a method post-deserialization.

You have some DTO class that is your system typed idea about the structure of the JSON -- this class is quite useful as an implicit documentation, but it really has to stay internal to the boundary. You will use an autodeserializer to such class and then you will continue by constructing real object from deserialized data that can be presented to the rest of the application. During such construction you can validate state and return errros.

This step can be eased by some validating attributes on the boundary DTO properties, but there is always some custom logic that describes what is acceptable and what is not.

[0] http://www.grpc.io/

The problem with this approach is that you've completely abdicated the power of the type system to ensure that your objects are valid. What happens if a field is missing from the JSON? Well, that field just becomes null. So now you have one of two options:

1) Write highly defensive code with null-checks everywhere. This is a pain to write, a pain to read, and almost impossible to get right and actually prevent null pointer exceptions. This is a nightmare. Switching to a null-safe language like Kotlin doesn't really help you beyond making sure that you actually code in all the null checks - the code is still ugly and a pain to maintain.

2) Call a (potentially) expensive verification method at the beginning of each method call for your object. This is less error prone than having null checks everywhere, but it's not much of an improvement. Because verification happens not at object creation time but rather when it's used, you'll find yourself with a verification exception at the entrance to some business logic where the JSON was passed to your system a week ago, immediately stored in a schema-less ORM, retrieved now, so you kind of have an idea that you have some client which didn't populate the field, but you have no idea which of the many, myriad versions of the client is responsible. So now you're fucked, and you're doubly fucked if you're losing data because of it.

Or you could just take advantage of type safety and write immutable object factories which refuse to instantiate invalid objects. Then you can write clean code using objects which you know must be valid because of type system guarantees. Libraries like immutables.github.io make this a piece of cake.

Not the case. I successfully used Jackson combined with Lombok to achieve some really nice DRY class definitions that Just Worked with Jackson. It took a little figuring out and a couple bugfixes to Jackson but it worked. That said, part of the hassle was that I insisted on being able to do this with @Wither so we could have the objects be immutable too.

Then you can write stuff roughly like

@Value public final class Thing { String name; int age; boolean boiling; }

Although IIRC I had to use some other random set of lombok annotations instead of @Value to get it to work right with Jackson (this was a while ago, don't recall details)

> The problem with this approach is that you've completely abdicated the power of the type system to ensure that your objects are valid.

Yeah, this was a big problem with my approach. The other choice would have been to use @Builders instead of @Withers. Then you get a little more boilerplate (having to type .build()), but you can guarantee the built objects meet consistency requirements. (In retrospect, I doubt I chose the right tradeoff there)

0 - https://github.com/google/gson/issues/61 1 - http://static.javadoc.io/com.fasterxml.jackson.core/jackson-...

public List<CustomerModel> Get(SearchRequest request) { ..... return customers; }

Somewhere else in the (configurable) pipeline the framework can decide how to deserialize the SearchRequest and how to serialize the List<customer> based on the Accept-Header.

(CustomerModel/Request would not be business objects. They would only be used to on the API layer)

As far as validation. You could just put attributes on the properties of the Request like [Required] and they would automatically be validated before your Get method is called. Of course if the types don't match, the framework would send the appropriate error.

Gson does not require this. The following class will serialize and deserialze fine with Gson:

    class Example {
      private final int foo;
      private final String bar;
    
      private Example(final int foo, final String bar) {
        this.foo = foo;
        this.bar = bar;
      }
    }

I've used Gson fine with Scala case classes.

Although now I just use Scala JSON libraries, which do not suffer from the two problems you list at all.

On the other hand, the fact that there's no runtime typecheck renders static analysis somewhat impotent when it comes to the result of a network call.

For instance, if your image compression application throws out my EXIF data that it doesn't understand, I'm going to be pissed. (Unless you give me an option to preserve it.)

True, although the OP seems to be advocating having your app pretty much ignore serialization altogether in favor of object-oriented design. In particular the author objects to use of dictionaries and lists instead of objects.

It is true that if you're designing an application with a json api in mind, you're likely to stick with the data structures that are easiest to serialize.

Personally, I started writing programs that way before json became so common. I did it simply to take full advantage of the native data structures and to avoid prematurely confining myself into an object hierarchy that wasn't a good fit for the problem domain. It also winds up making code more generic and easier to rewrite in a different language if necessary (for example, moving server-side code to client javascript).