This looks awesome. As someone working in support for a wide array of Linux Apps, and data dumps from customers where I have no access to the system, plus I also write or backport bug fixes to all sorts of random software, I often want to do this kind of crazy stuff. With exactly these kinds of artefacts.

Thank you for your reply.Yes, I was asking about new data sources, so for example if I wanted to add Github to query my GH issues with Musoq, how I would do that.Great, I&#x27;ll check out the links!

You can query with --format [raw|csv|json|interpreted_json]This will output pure json or csv. This way you can use other tools like jq, grep, csvtoolkit or whatever you need further process your data.to dig deeper, just look at: <a href="https:&#x2F;&#x2F;github.com&#x2F;Puchaczov&#x2F;Musoq.CLI">https:&#x2F;&#x2F;github.com&#x2F;Puchaczov&#x2F;Musoq.CLI</a>After some thought: you could have also asked whether it&#x27;s possible to add new data sources that you need to query, and the answer is of course yes! It&#x27;s actually quite simple and there are many examples. Each data source tool is just a plugin implementing the appropriate interfaces. You can look at some example projects and see how they implement their data source here:
<a href="https:&#x2F;&#x2F;github.com&#x2F;Puchaczov&#x2F;Musoq.DataSources">https:&#x2F;&#x2F;github.com&#x2F;Puchaczov&#x2F;Musoq.DataSources</a>

Very cool!How does this interface with the different tools and how would one add another tool for it to operate on?I started on something similar last year which was just a simple bash script to interact with things like osquery. Alas it was too buggy for what I wanted to do and it&#x27;s paused indefinitely for now.

Thanks!
It would be great to ask what kind of usage you have in mind? I&#x27;ve currently been using the Roslyn plugin mainly with the engine repository itself because it helps me extract queries from tests, I use it in combination with GPT, which helps me create documentation. In the longer term, I would like this plugin to help me refactor some problematic code fragments, but for that I will need to further develop the Roslyn integration itself.

Hey I like your project, earned a star from me! When time allows, will take it on a test drive to see how exactly it works with Roslyn&#x2F;C# data. My C# solution has grown to about 80 projects so it would be good test.

That helps, thanks! It is all about the framing. I love these kinds of tools and have been using a combination of them together with nushell, but it is a road less travelled it seems. All the more reason to evangelise your tool :) keep up the good work.

I see it a bit that way: while osquery focuses on OS-level queries and steampipe on cloud services, Musoq is more developer-centric, I&#x27;m using it like swiss army knife for various thing, something like sed or grep. You can ofcourse create plugins that covers what mentioned tools do or even, use that tools as a plugin for musoq but in general - I&#x27;m not going to compete with any of them, I&#x27;m filling my own niche - flexible developer tool.

Where would you place this between osquery and steampipe? It seems to borrow concepts on both sides, but I&#x27;m not sure how it could not be plugin for either.

You actually don&#x27;t need any special type system or complex infrastructure for this. Each data source can handle its own data representation and operations.
For GIS data, you could create a plugin that naturally handles spatial operations. Here&#x27;s how it could work:select 
 Id,
 DistanceBetween(s.FromPoint(-73.935242, 40.730610)) as Distance,
 s.IsBetween(s.FromPoint(-73.935242, 40.730610)) as IsInArea
from #gis.shapefile(&#x27;map.shp&#x27;) sYou could even combine it with other data sources. For example, if you have geometry data in a CSV:select 
 sfg.DistanceBetween(sfg.FromPoint(..., ...)) 
from #csv.file(...) c 
cross apply #gis.ShapeFromGeometry(c.Geometry) sfg

in postgresql syntax for data types is stored in the database. how would this tool parse gis expressions, for example?

Do you have something specific in your mind? I would love to hear that! Unexpected approach is what driving me usually

Ironically I think I&#x27;d rather query a database with anything other than SQL ¯\_(ツ)_&#x2F;¯

Hey, For those of you who don&#x27;t know my little tool Musoq, I wanted to introduce it as a small tool that allows you to query with SQL-like syntax without any database.It allows you to query various things from niche ones like CAN DBC files, weird ones like C# code, interesting ones with Git querying to regular stuff like CSV, TSV and various others.I am quite a bit experimenting with various things so I&#x27;m hybridizing the engine with LLMs or doing other weird stuff that are more or less practical :-)I wanted also to share some recent developments in this little project as I hope it might be interesting to some of you.New Experimental Plugins:
* Git Plugin (Beta): I&#x27;ve been working on Git repository querying - managed to test it on the EF Core repo (16k commits) and it seems to work okay
* Roslyn Plugin (Beta): Added basic C# code analysis capabilitiesFor the very first time:
I&#x27;ve extended CROSS APPLY to use computed results as arguments! Now the operator can use values from the current row as inputs. Here&#x27;s an example:<pre><code> SELECT
 f.DirectoryName,
 f.FileName
 FROM #os.directories(&#x27;&#x2F;some&#x2F;path&#x27;, false) d
 CROSS APPLY #os.files(d.FullName, true) f
 WHERE d.Name IN (&#x27;Folder1&#x27;, &#x27;Folder2&#x27;)
</code></pre>
After another pack of fixes I&#x27;m finally able to query multiple git repositories AT ONCE!<pre><code> with ProjectsToAnalyze as (
 select
 dir2.FullName as FullName
 from #os.directories(&#x27;D:\repos&#x27;, false) dir1
 cross apply #os.directories(dir1.FullName, false) dir2
 where
 dir2.Name = &#x27;.git&#x27;
 )
 select
 c.Message,
 c.Author,
 c.CommittedWhen
 from ProjectsToAnalyze p cross apply #git.repository(p.FullName) r 
 cross apply r.Commits c
 where c.AuthorEmail = &#x27;my-email@email.ok&#x27;
 order by c.CommittedWhen desc
</code></pre>
Under the Hood:
- Added a Buckets feature for memory management (currently just testing it with the Roslyn plugin)- Moved to .NET 8- Added CROSS&#x2F;OUTER APPLY operators- Made some improvements to error messages and runtime behaviorNew piping features:
I&#x27;ve been experimenting with piping capabilities:
* Image Analysis with LLMs:<pre><code> .&#x2F;Musoq.exe image encode &quot;image.jpg&quot; | .&#x2F;Musoq.exe run query &quot;select s.Shop, s.ProductName, s.Price from ...&quot;
</code></pre>
* Text Data Extraction:<pre><code> Get-Content &quot;ticket.txt&quot; | .&#x2F;Musoq.exe run query &quot;select t.TicketNumber, t.CustomerName ... from #stdin.text(&#x27;Ollama&#x27;, &#x27;llama3.1&#x27;) t&quot;
</code></pre>
* Data Source Combination:<pre><code> { docker image ls; .&#x2F;Musoq.exe separator; docker container ls } | .&#x2F;Musoq.exe run query &quot;...&quot;
</code></pre>
I&#x27;m working on comprehensive documentation:
I encourage you especially to look at section &quot;Practical Examples and Applications&quot; and &quot;Data Sources&quot; where you can look at all the tables the tool currently provides. &lt;<a href="https:&#x2F;&#x2F;puchaczov.github.io&#x2F;Musoq&#x2F;" rel="nofollow">https:&#x2F;&#x2F;puchaczov.github.io&#x2F;Musoq&#x2F;</a>&gt;Other Changes:- Made some improvements to OS and Archive data sources (OS can now query metadata like EXIF)- Added a few fields to CAN DBC plugin- Command outputs can now be used as inputs for queriesI&#x27;m hoping to:- Improve stability and add more tests- Flesh out the documentation- Work on package distribution (Scoop, Ubuntu packages)- Share some examples of source code querying with RoslynIdeas for later:- WHERE robust analysis and optimizations- DISTINCT operator implementation- PROTOBUF schema support- Performance improvements- Query parallelization- Recursive CTEs- SubqueriesI&#x27;d really appreciate any thoughts or feedback!The documentation section where I write a short analysis of EF Core with git plugin: &lt;<a href="https:&#x2F;&#x2F;puchaczov.github.io&#x2F;Musoq&#x2F;practical-examples-and-applications&#x2F;analysis-of-the-efcore-repository.html" rel="nofollow">https:&#x2F;&#x2F;puchaczov.github.io&#x2F;Musoq&#x2F;practical-examples-and-app...</a>&gt;

Show HN: Musoq – Query Anything with SQL Syntax (Git, C#, CSV, Can DBC)