Universal checks for every single solution?

[angelikatyborska]

As I mentor, I keep seeing some reoccurring problems with solutions that are exercise-independent and could, at least partially, be detected automatically.

A code snippet from a real recent solution:

@spec reverse(list) :: list
defp reverseHelper([], reversed) do
  reversed
end
defp reverseHelper([x|xs], reversed) do
  reverseHelper(xs, [x|reversed])
end

def reverse(l) do
  reverseHelper(l,[])
end

The analyzer could detect and give a suggestion to the user that:

• @spec for a function should be placed directly above that function definition, there shouldn't be a different function definition in between
• function (and variable) names should use snake_case

More opinionated, but we could also check if each public function has a @spec and @doc and nudge students to add them (for practice exercises only).

Without parsing the AST, we could check if the code is intended with spaces or with tabs.

I think in particular the camelCase and indenting are worth doing in the analyzer, because they're hard no-nos in community Elixir code, but they're annoying details that are pretty uncomfortable to point out to a new comer. It would be nicer if a machine did that 🙂.

[neenjaw]

I like this idea in moderation. The casing stuff I think is important enough to warrant extensions. The indenting, I would rather point people to mix format rather than wade into a discussion of tabs vs spaces.

[neenjaw]

We could create a mix format microservice (picoservice?) which accepts a chunk of code and "formats it" and returns it to the editor :P

[angelikatyborska]

The indenting, I would rather point people to mix format

In which way would you point people? From the context I'm guessing NOT by having an analyzer comment? Would you expect mentors to tell students that?

We could create a mix format microservice (picoservice?) which accepts a chunk of code and "formats it" and returns it to the editor :P

That would be great because students in the web editor otherwise won't have access to the auto formatting :D but I have a hunch it wouldn't be part of v3.0...

Now that the base work for having common checks is in there, I'm going to create separate tickets for separate checks so that other people can work on them too.

[angelikatyborska]

Added three so far that I was personally sure about. Feel free to add more or dispute specific ones 🙂

#103
#104
#105

[jiegillet]

How about adding checks for IO.inspect and IO.puts?

[neenjaw]

How about adding checks for IO.inspect and IO.puts?

If this were to be implemented, I would suggest to it be informational at most since there might be a specific reason they left it in. (Partial optimized solution in order to elicit/engage mentor feedback, etc)

[jiegillet]

Good point. I once had a weird process-related bug that depended on a IO.puts("") to work, I still submitted that to ask a mentor.

[angelikatyborska]

@jiegillet I would be fine with an informational comment telling the student to try to remove usages of IO.inspect and IO.puts after they finished debugging. It is a common problem. But you would need first to add a mechanism for specific "common checks" to be excluded from specific exercises. There are two concept exercises, for io and for file, that require using those two functions:

https://github.com/exercism/elixir/blob/main/exercises/concept/rpg-character-sheet/.meta/exemplar.ex
https://github.com/exercism/elixir/blob/1bef66fc55381ea44d02df36a7cef855e780a93b/exercises/concept/newsletter/.meta/exemplar.ex

[jiegillet]

How about adding informational checks about using @spec for non-private functions? And maybe @moduledoc? This may be too preachy :)
EDIT: I'm an idiot, those already exist.

[angelikatyborska]

There's a check for two-fer for those but not much else. Using @spec, @moduledoc and and @doc is a personal choice in my opinion. It's definitely helpful in bigger projects, but still optional. I wouldn't bother people with those in the analyzer. Especially because using them is a concept exercise on its own which means we cannot "require" people to know about them before doing that exercise.

[jiegillet]

We could create a mix format microservice (picoservice?) which accepts a chunk of code and "formats it" and returns it to the editor :P

That would be great because students in the web editor otherwise won't have access to the auto formatting :D but I have a hunch it wouldn't be part of v3.0...

Re-reading the thread, I think the idea was to add a common check that runs mix format on the student's code, and if there is a difference, dump the formatted code in an informational comment with "We noticed that your code didn't use mix format, a standard formatting tool in the community, here is what the formatted code would look like: ....". We could even have a celebratory comment that says "Your code is formatted perfectly, well done." is the code is well formatted.

[angelikatyborska]

I felt comfortable telling everyone about mix format when I knew that everyone is using the CLI (and only if their solution was horribly misformatted). Now with the web editor, getting a comment about a formatting problem, even if it's really tiny, might be annoying and hard to fix. The value of mix format is that you don't have to think about formatting, it just happens automatically. That analyzer comment would be the opposite of "not have to think about".

Could there be a way of only showing this comment if the formatting differences are... "big enough" (very specific term, I know :D)?

[jiegillet]

You're right, it made sense for the CLI, not for the web editor.
Quantifying differences... Mmh... Maybe it can be done? Maybe something like if the diff if more than 15-20% of the total character count?

Another idea I had was coming back to the compiler warnings. I revisited an older solution that a mentor commented on because I had a compiler warning. Now there aren't any ways to get to those warnings on the web editor. If there was a way to get compiler warnings in the analyzer, we could show :informational messages.