• 44 Posts
  • 434 Comments
Joined 1 year ago
cake
Cake day: June 9th, 2023

help-circle

  • Multi threading is parallelism and is poised to scale to a similar factor, the primary issue is simply getting tensors in and out of the ALU. Good enough is the engineering game. Having massive chunks of silicon laying around without use are a mach more serious problem. At the present, the choke point is not the parallelism of the math but actually the L2 to L1 bus width and cycle timing. The ALU can handle the issue. The AVX instruction set is capable of loading 512 bit wide words in a single instruction, the problem is just getting these in and out in larger volume.

    I speculate that the only reason this has not been done already is because pretty much because of the marketability of single thread speeds. Present thread speeds are insane and well into the radio realm of black magic bearded nude virgins wizardry. I don’t think it is possible to make these bus widths wider and maintain the thread speeds because it has too many LCR consequences. I mean, at around 5 GHz the concept of wire connections and gaps as insulators is a fallacy when capacitive coupling can make connections across all small gaps.

    Personally, I think this is a problem that will take on a whole new architectural solution. It is anyone’s game unlike any other time since the late 1970’s. It will likely be the beginning of the real RISC-V age and the death of x86. We are presently at the age of the 20+ thread CPU. If a redesign can make a 50-500 logical core CPU slower for single thread speeds but capable of all workloads, I think it will dominate easily. Choosing the appropriate CPU model will become much more relevant.


  • Mainstream is about to collapse. The exploitation nonsense is faltering. Open source is emerging as the only legitimate player.

    Nvidia is just playing conservative because it was massively overvalued by the market. The GPU use for AI is a stopover hack until hardware can be developed from scratch. The real life cycle of hardware is 10 years from initial idea to first consumer availability. The issue with the CPU in AI is quite simple. It will be solved in a future iteration, and this means the GPU will get relegated back to graphics or it might even become redundant entirely. Once upon a time the CPU needed a math coprocessor to handle floating point precision. That experiment failed. It proved that a general monolithic solution is far more successful. No data center operator wants two types of processors for dedicated workloads when one type can accomplish nearly the same task. The CPU must be restructured for a wider bandwidth memory cache. This will likely require slower thread speeds overall, but it is the most likely solution in the long term. Solving this issue is likely to accompany more threading parallelism and therefore has the potential to render the GPU redundant in favor of a broader range of CPU scaling.

    Human persistence of vision is not capable of matching higher speeds that are ultimately only marketing. The hardware will likely never support this stuff because no billionaire is putting up the funding to back up the marketing with tangible hardware investments. … IMO.

    Neo Feudalism is well worth abandoning. Most of us are entirely uninterested in this business model. I have zero faith in the present market. I have AAA capable hardware for AI. I play and mod open source games. I could easily be a customer in this space, but there are no game manufacturers. I do not make compromises in ownership. If I buy a product, my terms of purchase are full ownership with no strings attached whatsoever. I don’t care about what everyone else does. I am not for sale and I will not sell myself for anyone’s legalise nonsense or pay ownership costs to rent from some neo feudal overlord.






  • Yeah this has been my experience too. LLMs don’t handle project specific code styles too well either. Or when there are several ways of doing things.

    Actually, earlier today I was asking a mixtral 8x7b about some bash ideas. I kept getting suggestions to use find and sed commands which I find unreadable and inflexible for my evolving scripts. They are fine for some specific task need, but I’ll move to Python before I want to fuss with either.

    Anyways, I changed the starting prompt to something like ‘Common sense questions and answers with Richard Stallman’s AI assistant.’ The results were remarkable and interesting on many levels. From the way the answers always terminated without continuing with another question/answer, to a short footnote about the static nature of LLM learning and capabilities, along with much better quality responses in general, the LLM knew how to respond on a much higher level than normal in this specific context. I think it is the combination of Stallman’s AI background and bash scripting that are powerful momentum builders here. I tried it on a whim, but it paid dividends and is a keeper of a prompting strategy.

    Overall, the way my scripts are collecting relationships in the source code would probably result in a productive chunking strategy for a RAG agent. I don’t think an AI would be good at what I’m doing at this stage, but it could use that info. It might even be possible to integrate the scripts as a pseudo database in the LLM model loader code for further prompting.










  • We have no data for an Earth analog around a G-type star, like absolutely nothing. I highly doubt there is some universal life around such a star, but out of a sample size of 1, who could rule them out? Kepler was barely supposed to be able to survey at this resolution, but totally failed at that objective. They claimed success for politically criminal reasons, but go look at the actual data and you’ll see the random noise they cherry picked to make that claim and how they are massive outliers from the rest of the data. None of those data points are remotely scientifically relevant or taken seriously. No other survey to date has come close to an Earth like resolution.

    Researching for my book, there several G-type stars within 7 parsecs. I find them most interesting, but I do not believe complex life is likely anywhere in this galaxy at the present point in time.


  • There were some talks on Harvard’s CfA colloquium that went into the evidence for a prior generation of Jovian moons and that these were likely larger. I’ve long speculated that this event is by far the most likely to produce a Theia like object on a potential Earth collision course.

    Earth’s hotspots are from a paper this year on the mantle anomalies and ongoing research that correlates them. I watch a range of qualified academic sources with valid and current credentials. This is like my casual entertainment. I don’t care to argue, act like chimps in a zoo throwing party favors, their academic equivalent, or the bottom tier of Dante’s Reddit. You can find most of this through Anton Petrov and Fraser Cain in the last few years. Moat of these ideas are published and the rest are abstraction and speculation that fits the more broad strokes of what I have seen.

    I really don’t care who is right or wrong. I enjoy my abstractions in observation from the sidelines. I do not take sides except those that seem more compelling and place no value on tenure or those that appear to build blind careers on poor intuition that makes no sense outside of a narrow frame of reference.

    If you take offense at such a casual interest I apologize. Feel free to block me. I do the same to rudeness online as I do in public to my face.


  • j4k3@lemmy.worldtoAsk Lemmy@lemmy.worldDo aliens exist in our planet?
    link
    fedilink
    English
    arrow-up
    12
    arrow-down
    1
    ·
    edit-2
    12 days ago

    The only remote chance is panspermia. There is absolutely no chance that complex life exists within 1 astronomical unit of Earth, or within a few parsecs for that matter.

    Simple life is likely common, but not complex. The most likely large filter for Earth, appears to be the Theia collision that lead to the moon’s formation, Earth’s hotspots, a likely source of nitrogen and water of the correct isotopes, and the vital plate tectonics that enable elemental cycles beyond anything seen on other terrestrial worlds.

    The real question is not what makes Earth unique; it is what makes Venus and Mars so similar, any why Earth is not an average intermediate. The obvious answer is Luna.