Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The future of jsforce #1360

Closed
8 of 9 tasks
cristiand391 opened this issue Sep 18, 2023 · 29 comments
Closed
8 of 9 tasks

The future of jsforce #1360

cristiand391 opened this issue Sep 18, 2023 · 29 comments

Comments

@cristiand391
Copy link
Member

cristiand391 commented Sep 18, 2023

Hey folks! We have some exciting news to share.

After talking with @stomita , we've agreed the Salesforce CLI team will help maintain jsforce!
jsforce is a core component of the Salesforce CLI, Salesforce Extensions for VSCode, DevOps Center and a bunch of other Salesforce tools and we hope we can help Shinichi keep the project alive while delivering new features.

We know there hasn't been much activity triaging issues and PRs except for a few specific bug fixes/features, you can expect us to be more active in the repo while we work towards the v3 GA.

Dropping support for old node and browsers

jsforce v2 has been in beta for almost 3 years supporting node >= v8 and some old browsers (polyfills).
To avoid breaking projects already using v2 we plan to skip and go straight to v3 for the GA, requiring node >= 18 (current LTS) and modern browsers.

Browser usage

While we don't use jsforce in browsers (CLI and others run via node), we are committed to keep jsforce an isomorphic library.
We've already improved our CI test setup to run browser tests on latest Chrome, updated webpack/babel deps and fixed some flaky tests to ensure a safe path while we finish the last bits of work.

We will also remove the edge versions (/dist) these were already outdated when using a CDN. If you wish to continue using them, you can continue using the CDN versions here and the jsforce npm package includes these so they are available in modern CDNs that serve from npm.

Remaining work

@mshanemc mshanemc pinned this issue Sep 18, 2023
This was referenced Sep 18, 2023
@gmaliar
Copy link

gmaliar commented Sep 18, 2023

@cristiand391 how can we be helpful? The remaining work is unclear in how we can participate.

@techieshark
Copy link

we hope we can help Shinichi keep the project alive while delivering new features.

This is fantastic news! Very cool that Salesforce is investing in open source. As a Salesforce user/customer, very glad to see this happening as this library is very important to us as well.

Would love to know if Salesforce will be sponsoring Shinichi or any other open source developers?

Also curious how Salesforce will prioritise work - will it be any/all of the open issues, or issues with most activity, or PRs the community puts forward, or more tightly focused on features that more closely align with the CLI team's needs? I imagine knowing this might help I or others know how best to collaborate.

@cristiand391
Copy link
Member Author

@gmaliar

The CLI team is currently working on the listed work above. I wouldn't feel comfortable asking people to contribute while we still have a ton of PRs to triage. That said, if anyone is interested in working on something, just ping me in the issue/pr and we can discuss any fix/feature request before submitting a PR 👍🏼

@cristiand391
Copy link
Member Author

@techieshark

Would love to know if Salesforce will be sponsoring Shinichi or any other open source developers?

Salesforce has a program called FOSS Fund where employees can nominate and vote on projects to receive $10K, this is done every quarter. I know jsforce won it a few years ago (before I joined Salesforce) but seems it ended up rejected, see:
https://opencollective.com/salesforce/expenses/40748

@stomita do you remember what happened with that grant?

Anyway, the one for this quarter is closed but I would expect jsforce could be nominated for the next one (will be checking with the program manager since it's jsforce isn't owned by Salesforce but now we help maintain it).

I'm not aware of other Salesforce program to fund individual developers.

Also curious how Salesforce will prioritise work - will it be any/all of the open issues, or issues with most activity, or PRs the community puts forward, or more tightly focused on features that more closely align with the CLI team's needs? I imagine knowing this might help I or others know how best to collaborate.

We'll probably be fixing a few bugs as we triage issues and PRs targeting v2.
Features/bug fixes that align with our team's need would be our priority but as part of maintainers we'll also review community contributions and/or work on stuff not related to the CLI as much as we can.
In the past other internal teams offered to work on new features and we helped them reviewing/testing (Heroku added support for the Bulk V2 api in jsforce v2).

@FeiPengDev
Copy link

FeiPengDev commented Oct 16, 2023

Sorry to disturb you by asking questions here. I would like to ask if there is any progress on the following issues?
I am using webpack5 now and I still get errors as below
#1222
#1270

@cristiand391
Copy link
Member Author

Hey @FeiPengDev, sorry for the late reply. If you still have issues with webpack 5 see this comment: #1222 (comment)

@cristiand391
Copy link
Member Author

cristiand391 commented Nov 7, 2023

quick update:
node-only plan: #1374

feedback is welcome 👍🏼

I'll post here once the first node-only release is up on npm (right now we published an empty pkg to reserver the name)

@cristiand391
Copy link
Member Author

Alright, a few hours ago we published the first v3 releases under the next dist tag on npm:

jsforce:
https://www.npmjs.com/package/jsforce/v/3.0.0-next.1

jsforce-node:
https://www.npmjs.com/package/@jsforce/jsforce-node/v/3.0.0-next.1

I'll be on PTO next week but will start putting a few of the planned breaking changes for v3 here when I'm back.

@pavitra-infocusp
Copy link

Where can I learn about v3 API changes? Link to docs?

@john-bflow-mcgowan
Copy link

john-bflow-mcgowan commented Jan 16, 2024

@cristiand391: first of all, thanks for working on this awesome package! It's made approaching a new integration a lot more digestible.

Like @pavitra-infocusp I'm also curious to know about documentation for V2 and V3. Does the currently hosted documentation at https://jsforce.github.io/ reflect the changes already in the V2 beta version? I'm unable to find any reference to a jsforce version number other than what is mentioned in specific package method documentation (as seen in the Operation for Multiple Records section for example) -- but i'm assuming that the live docs refer to the latest stable release 1.11.1, and not V2.

I see the TODO for Documentation (linked below) is marked as complete. Is the command referenced a way to generate docs locally?

[X] Documentation equivalent to the current [jsforce.github.io](http://jsforce.github.io/) (http://jsforce.github.io/). PR https://github.com/jsforce/jsforce/pull/1362

@cristiand391
Copy link
Member Author

Hey @john-bflow-mcgowan , sorry for the late reply 🙏🏼

Does the currently hosted documentation at https://jsforce.github.io/ reflect the changes already in the V2 beta version?

no, we had those under https://jsforce.github.io/jsforce/doc/ for v2 but had to revert back to v1 docs because of the current docs setup.

I see the TODO for Documentation (linked below) is marked as complete. Is the command referenced a way to generate docs locally?

yeah, those generate API docs using typedoc, the end result is smt like this:
https://forcedotcom.github.io/sfdx-core/

but that's just the API reference, the document section will also be updated for v3 changes
https://jsforce.github.io/document/

@pavitra-infocusp
Copy link

@cristiand391 Is there any expected release date for v3?

@tehnrd
Copy link

tehnrd commented Apr 8, 2024

I would like to request that the base package be truly isomorphic, as it is not today. With many frameworks that do server-side rendering (SSR), like Sveltekit and Next.js, this library does not work. These run javascript on the server but do so with a browser runtime (not node) for rendering, so if you include node modules like Buffer, it will fail to build and run.

For example, if you import jsforce like this:
import jsforce from 'jsforce';

And then when you attempt to build for a serverless environment like Cloudflare Pages, you will receive the following error:

✘ [ERROR] Could not resolve "buffer"

    node_modules/core-util-is/lib/util.js:103:27:
      103 │ exports.isBuffer = require('buffer').Buffer.isBuffer;
          ╵                            ~~~~~~~~

  The package "buffer" wasn't found on the file system but is built into node. Are you trying to
  bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

As this library is rebuilt to use modules, it would be good to import node-specific libraries with the following import as many hosting platforms will automatically polyfill these:

// Do this:
import { Buffer } from 'node:buffer';

// Do not do this:
import { Buffer } from 'buffer';

@cristiand391
Copy link
Member Author

@tehnrd which version of jsforce are you using?

that import comes from core-util-is which I don't see being shipped in prod. versions.

These run javascript on the server but do so with a browser runtime (not node) for rendering, so if you include node modules like Buffer, it will fail to build and run.

Yeah, by isomorphic we mean node + browsers for now.
Not sure if we would "officially" support new runtimes now, but maybe if it's not too much effort we can help with stuff like you mentioned about imports, feel free to open new issues if something doesn't work with the node-compat mode 👍🏼 .

IIRC, using node:* imports triggered some errors in the build/bundle process with webpack but I can check again.

@tehnrd
Copy link

tehnrd commented Apr 8, 2024

I tried 1.11.1 and 3.0.0-next.3 but I need to do some deeper debugging.

@cristiand391 ya, I will spin up a new issue. It think (haven't investigated too much) the issue is that there is code directly referencing modules like Buffer (and others) object globally but this isn't recommended in the official node docs and it should be explicitly imported with the node: prefix. This way it can easily be polyfilled when doing builds across runtimes.

While the Buffer class is available within the global scope, it is still recommended to explicitly reference it via an import or require statement.

const { Buffer } = require('node:buffer');

// or

import { Buffer } from 'node:buffer';

https://nodejs.org/api/buffer.html#buffer

@satanshiro
Copy link

the newest releases are not part of the -next- versioning format yet the prerelease tag is still existent.
depandabot has been recommending to upgrade for this reason.
is the pre release over?

@GBvanDam
Copy link

I believe Connection.query(soql) is the only way to retrieve results in the format containing both totalSize, nextRecordsUrl and records. As far as I know this function only accepts a soql query in string format. I have been looking for a way to retrieve this response structure whilst using the conditions in JSON format (just like in conn.sobject("Contact").find({ ...conditions }).

If I did not miss any different method to accomplish this whilst using the conditions in object style, I think there are several possibilities:

  1. open up the Query class such that we can make calls like new Query(conn, { ...conditions }, locator).toSOQL() (currently this class is protected)
  2. support conditions in object style in the Connection.query() call
  3. or anything that I'm currently overlooking

Could you enlighten me, or could this become a new feature?

@stomita
Copy link
Contributor

stomita commented May 20, 2024

@GBvanDam

open up the Query class such that we can make calls like new Query(conn, { ...conditions }, locator).toSOQL() (currently this class is protected)

Actually the find() method returns Query object so you can do the following:

const soql = await conn.sobject("Contact").find({ ...conditions }).toSOQL();
const queryResult = await conn.query(soql);

@GBvanDam
Copy link

Thank you @stomita I clearly oversaw that option!

@GBvanDam
Copy link

One more question on jsforce, as far as I can see uploading files or blobs (especially the blob upload way with the highest file size limit) is not part of the jsforce scope. Is that intentionally, or am I overseeing something once again?

I'm trying to accomplish https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_sobject_insert_update_blob.htm using nodejs + axios, but are having trouble with generating the correct form data (either the formdata does not correctly represent the entity_document json, or the content type is switched to something other then form-data using nodejs + axios in my case), I'm even not successful in replicating https://salesforce.stackexchange.com/questions/222580/upload-large-contentversion-records in nodejs with axios (or any other fetch mechanism)

@cristiand391
Copy link
Member Author

One more question on jsforce, as far as I can see uploading files or blobs (especially the blob upload way with the highest file size limit) is not part of the jsforce scope. Is that intentionally, or am I overseeing something once again?

@GBvanDam yes, this isn't implemented yet in jsforce.

have you seen this?
#1426 (comment)

@GBvanDam
Copy link

@cristiand391 thanks for your reply! Indeed I have seen the base64 method, I have a version of it running on the ContentVersion endpoint I believe, but the base64 approach is limited to 25MB (or 50?) and as far as I know the conversion to base64 is quite resource intensive in terms of RAM as compared to simply streaming a buffer (although my knowledge on that topic is not that good)

@Joemusarurwa
Copy link

@cristiand391 When may we expect a release date for v3?

@cristiand391
Copy link
Member Author

@Joemusarurwa pretty much all the work is done now, we'll wrap up last details about jsforce v1 (mostly about maintenance) and get back last week.

@romainbenard
Copy link

Hi @cristiand391 any news about the release date ? :)

@cristiand391
Copy link
Member Author

we plan to do it next week. FYI @stomita, I've been trying to reach out to you via email to get your thoughts 🙏🏼

@matthewblewitt
Copy link

matthewblewitt commented Jul 10, 2024

@cristiand391 Is there any upgrade documentation with breaking changes for v1 to v3?

@cristiand391
Copy link
Member Author

@matthewblewitt yes, see the MIGRATING*.md docs in the v3 branch:
https://github.com/jsforce/jsforce/tree/3.0

@cristiand391
Copy link
Member Author

v3 is GA 🚀
#1529

will let this issue pinned for a bit, feel free to continue discussions in the announcement post.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests