Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Towards 0.16.0 #688

Closed
viktorTarasov opened this issue Feb 24, 2016 · 88 comments
Closed

Towards 0.16.0 #688

viktorTarasov opened this issue Feb 24, 2016 · 88 comments

Comments

@viktorTarasov
Copy link
Member

I think it's time to prepare and publish the next major release.
The 0.16.0-pre1 is already created, so, let us continue this initiative.

I propose the end of the next week as a limit date to close (or to decide to postpone) the pending issues and PRs, or other issues that are not yet published.
And so, create first RC at the beginning of the week after next .

What will you say?

@dengert
Copy link
Member

dengert commented Feb 24, 2016

I will be on vacation next week, reading e-mail but can not test anything till 3/8/2016.

What changes are you expecting to add between now and then?

On 2/24/2016 7:46 AM, viktorTarasov wrote:

I think it's time to prepare and publish the next major release.
The 0.16.0-pre1 is already created, so, let us continue this initiative.

I propose the end of the next week as a limit date to close (or to decide to postpone) the pending issues and PRs, or other issues that are not yet published.
And so, create first RC at the beginning of the week after next .

What will you say?


Reply to this email directly or view it on GitHub #688.

Douglas E. Engert [email protected]

@viktorTarasov
Copy link
Member Author

@dengert: ... can not test anything till 3/8/2016.

  • there is no strict release schedule;
  • until your date there will not be RC to test -- this period is to fix what has to be fixed/merged/.. into master before RC.

@dengert: What changes are you expecting to add between now and then?

From my side, for this period, there is no changes that are not currently in master ;
will try to resolve the existing PRs and issues, and see if there will be requests to include something else before train departure.

@frankmorgner
Copy link
Member

  1. Someone (else) should go through all defects reported by Coverity scan. I started to resolve them here
    Fix some problems reported by Coverity #676, but there are still many new issues since the last release
  2. Unfortunately I can't make much progress on Add Support for electronic signature with German ID card #611. Everything is fine and tested on Linux since last year. However, on Windows there seems to be an invalid free on card->applications on opensc.dll which I have no idea of how to fix (the memory address equals the allocated one and it has not been free'd before). I am suspecting a Windows specific dll-loading problem... Any help would be appreciated.

@viktorTarasov
Copy link
Member Author

Think that after decision about #705 and #704 the first RC of 0.16.0 will be created.

If you need to have resolved in next release the issues/PRs of middle complexity,
that have not been discussed recently -- it's time to mention them.

During this period will be fixing the evident coverity-scan issues.

@frankmorgner
Copy link
Member

For inclusion to the upcoming release, I extracted the generic features of #611 into #706

@frankmorgner
Copy link
Member

#694 should also be looked at for the next release. It still could need some improvements, as discussed in #655, but it works for now. @viktorTarasov could you have a look?

@viktorTarasov
Copy link
Member Author

@frankmorgner

could you have a look?

No, if it do not concern the common part,
or, maybe, as a last priority task.

@dengert
Copy link
Member

dengert commented Mar 18, 2016

@viktorTarasov @frankmorgner

The removal of the hot-plug slot, with #687 #704 renumbers the slots.

This could surprise many users who have scripts expecting to use --slot 1 or slot_1
and now have to use --slot 0 or slot_0 (or don't use the parameter if there is only one reader.)

@viktorTarasov
Copy link
Member Author

I understand, but:

  • hot plug do not work in all cases (not when inserted more then one token);
  • I do not get answer for question: when hot-plug is needed -- FF works as expected without it.
  • as for me, slot reserved for hot-plug, first in the list, in default configuration -- it's not normal.

@dengert
Copy link
Member

dengert commented Mar 18, 2016

I am not saying to keep the hot-plug. But the user experience will change and this should be documented.

On 3/18/2016 12:59 PM, viktorTarasov wrote:

I understand, but, as for me:

  • hot plug do not work in all cases (not when inserted more then one token);
  • I do not get answer for question: when hot-plug is needed -- FF works as expected without it.
  • as for me, slot reserved hot-plug, first in the list, in default configuration -- it's not normal.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub #688 (comment)

Douglas E. Engert [email protected]

@viktorTarasov
Copy link
Member Author

RC1 is ready to be tested. (Also on sourceforge)

Some auxiliary files still need to be finalized:

  • Release notes;
  • NEWS -- TODO section contains list of commits to be mentioned or not in NEWs;
  • XML doc sources for some tools (from recent PRs)

@viktorTarasov
Copy link
Member Author

viktorTarasov commented Mar 28, 2016

OK tests RC1:

  • IAS/ECC (Gemalto) with secure-messaging:
    • pkcs15 tools:
      • erase applications;
      • import, generation of keys, certificates into generic and protected applications;
      • sign, decrypt with key in generic and protected applications;
      • verify, change, unblock of PIN and SignPIN;
      • list and get PKCS15 objects.
    • pkcs11 :
      • with FF and icedove/thunderbird, linux/Win-Vista
        • generate key/import certificates Sign and Auth, import Encr pkcs12;
        • authentication, mail signature and encryption.
    • minidriver (with candidate to RC2 that includes Auxiliary, non-pkcs15, data in pkcs15 data types #728):
      • with IE and Microsoft Windows Mail:
        • X509 Enrollment - generate key, import cert;
      • authentication, mail signing, mail decryption.
      • TODO: Import PKCS12
  • 'CardOS v4.3B', 'Cryptoflex 16k', 'SetCOS 4.4.1 B'
    • pkcs15 tools:
      • re-initialize;
      • generate RSA-1024 key, import pksc12;
      • sign data, get pub. key
      • verify, change UserPIN
      • decrypt ('SetCOS 4.4.1 B')
  • Oberthur: 'AuthentIC v3.2.2', 'Cosmo-64 v5.1', 'ePass 2003':
    • pkcs15 tools:
      • re-initialize;
      • generate RSA key, import pksc12;
      • get cert, pub. key
      • sign, decrypt
      • verify, change UserPIN

@dengert
Copy link
Member

dengert commented Apr 8, 2016

@viktorTarasov When do you plan to merge opensc-0.16.0 branch into master?
I note that your #728 hits 36 modules and is based on master. The longer we keep separate branches the harder it will be to merge them.

@viktorTarasov
Copy link
Member Author

@dengert I developed and tested #728 on opensc-0.16.0 branch
(For a while on local repo. Will be pushed to github after some grace time for discussion of PR. )

PR #728 is not to be merged to master, but rather to be discussed.
(Yes, after reflections, I could make PR relative to opensc-0.16.0.)

RC2 has to be with these changes.

I will finish the QA testing of RC1 for IAS/ECC on WIndows (with code of this PR),
then some, not exhaustives, tests with the other cards that I have,
then wait a little bit for the testing results from other people,
cleanup release notes,
and then merge to master.

@dengert
Copy link
Member

dengert commented Apr 8, 2016

@viktorTarasov
I am not doing anything else with OpenSSL-1.1.0 until 0.16.0 is merged to master. I don't expect the OpenSSL-1.1.0 changes to be part of 0.16.0.

Are you planing on #728 being part of 0.16.0?

@viktorTarasov
Copy link
Member Author

@dengert

Are you planing on #728 being part of 0.16.0?

Yes that's my intention. Without it X509 Enrollment for IAS/ECC is not working.

@dengert
Copy link
Member

dengert commented Apr 8, 2016

OK.

@germanblanco
Copy link
Contributor

@frankmorgner, thank you for referencing #694.
It does not concern the common part, but what will it take to get it in then?
Will it be enough to get a review from e.g. @rickyepodery and @miguel-cv?
This change removes memory leaks and improves the baseline for the DNIe module in order to make a better use of the OpenSC SM framework in the next release.

@mouse07410
Copy link
Contributor

@viktorTarasov, @frankmorgner, @dengert could you please explain to me the status of OpenSC branches, particularly "master" vs "0.16.0"? I would think that "0.16.0" would be a "frozen" stable off-shoot from "master", but it appears that "0.16.0" has commits not merged into "master" itself...?

So what is the plan? And/or what am I missing?

@dengert
Copy link
Member

dengert commented Apr 19, 2016

Good question. I would expect that each RC would be merged into master.

On 4/18/2016 5:39 PM, Mouse wrote:

@viktorTarasov https://github.com/viktorTarasov, @frankmorgner https://github.com/frankmorgner, @dengert https://github.com/dengert could you please explain to me the status of OpenSC branches,
particularly "master" vs "0.16.0"? I would think that "0.16.0" would be a "frozen" stable off-shoot from "master", but it appears that "0.16.0" has commits not merged into "master" itself...?

So what is the plan? And/or what am I missing?


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub #688 (comment)

Douglas E. Engert [email protected]

@viktorTarasov
Copy link
Member Author

viktorTarasov commented Apr 19, 2016

For me branch 0.16.0 is the release branch (name may be misleading -- it can be changed), where are fixed the issues that appears during the tests.
This branch contains the RC tags. These tags are proposed for testing. Theses tags are not merged to master. At the end of QA master switched to final RC and tagged as official 0.16.0 .

There are no frozen branch, there are tags.

During the QA period to master are cherry-picked only fixes of outstanding issues, that are equally present in prepare-release branch.

@dengert
Copy link
Member

dengert commented Apr 19, 2016

I don't see the point in in using a branch name, if the intent is to merge it into master when you are done.
I don't see any changes being added to master during this process.
Changes like #739 is a global changes that affect all cards and readers, yet are being added after after 0.16.0 rc1.
Is this change so critical that it needs to be added to RC2?

Do we get more testing done using a RC branch vs adding the changes to master now?

But I am not in a position to do much testing or change the process. Thanks for working on the next release.

On 4/19/2016 3:16 AM, viktorTarasov wrote:

For me branch 0.16.0 is the release branch (name may be misleading -- it can be changed), where are fixed the issues that appears during the tests.
This branch contains the RC tags. These tags are proposed for testing. Theses tags are not merged to master. At the end of QA /master/ switched (or merged) into final RC, this merge will be tagged as
official /0.16.0/ .

There are no frozen branch, there are tags.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub #688 (comment)

Douglas E. Engert [email protected]

@viktorTarasov
Copy link
Member Author

Is this change so critical that it needs to be added to RC2?

Without #739, to restore init of CardOS, we have to revert 2e21163 .
(Other commits to release branch after rc1 concern the broken MD for IAS/ECC card, absence of control on growing MD log file, ...)

The general problem of current code in the reader-pcsc part is that

  • it uses the the reader's TLV property max-APDU-size in a wrong way (ext. APDU reported by PCSC);
  • by default all readers supposed to be able for extended APDU which is not the case. Cards, like CardOS, that pretend to support extended APDUs will not work with readers without extended APDU.

Do we get more testing done using a RC branch vs adding the changes to master now?

In master there is no commits that are not present in release branch.

I would like to tag the release branch with RC2 after the basic tests with the cards that I have. This tag will be proposed for the final testing.

@frankmorgner
Copy link
Member

I think @dengert 's point was that instead of pushing everything to the branch you could push it to master instead. Everything will end up there anyway. A dedicated branch only makes sense if you want to make a feature freeze so that some commits from master will not be included in this release. But since merging has been carefully recently in awareness of the upcoming release there was no such addition to master.

I also agree with @dengert that there were some major additions recently that still need time to be evaluated. I still have in mind the trouble we got last time...

@viktorTarasov
Copy link
Member Author

Everything will end up there anyway.

From this point of view I do not see much subject to debate.

Afais in OpenSC it's rare to have branch dedicated to fix or feature. There is no collaborative work -- all fixex and features are in the personal clones.

For me branch dedicated to release is something natural. I do not see substantial importance in different approach. Essential -- tag with official release somewhere in master.

@mouse07410
Copy link
Contributor

In master there is no commits that are not present in release branch.

It should be the other way.

I brought the question up because I saw significant divergence between master and 0.16.0, the latter having 28 more commits (60 files changed, according to GitHub today). If 0.16.0 is supposed to be a development branch where commits are tried and tested prior to being merged into the master - it's all good and well. If it is a release branch - then the fact that master is way behind the release is troublesome, at least to me.

@viktorTarasov
Copy link
Member Author

In release branch the bugs are found out during the tests, they are fixed and retested in the same, release, branch. So, release branch, in a most natural way, is ahead of master.

@mouse07410
Copy link
Contributor

mouse07410 commented Apr 26, 2016

In release branch the bugs are found out during the tests, they are fixed and retested in the same, release, branch...

OK, so it appears what you call "release" I call "development". Because to me it isn't a "release" (not even a "candidate") until it's tested and stable. ;)

Regardless, thanks for your efforts.

0.16.0rc? seems to work with my tokens and PIV cards. Update: not any more (see posts below)

@viktorTarasov
Copy link
Member Author

I call release the branch where the next release is prepared. The head of branch is not stable. Stable are (have to be) the tags.

When it was started I do not thought that there will be serious regression problems. So, by necessity, this branch has development features.

@viktorTarasov
Copy link
Member Author

I changed the name of release branch to towards-opensc-0.16.0.

@mouse07410
Copy link
Contributor

mouse07410 commented May 17, 2016

it's not installed. It is linked in statically

Somewhat better - but still, it sucks disk space and adds unnecessary complexity. I for one want my apps to use one OpenSSL - and I want it to be the one whose configuration (and version) I chose. I do not want somebody to stick 1.0.2-stable on my system. Not to mention a likely nice conflict when I get libp11 involved and use OpenSSL (my OpenSSL) to programmatically access tokens, which uses OpenSC that with your help would have a different idea of instances, formats, etc. Not to mention that some of my builds are on disconnected machines...

Why can't you use the installed OpenSSL like the master, 0.15.0, and 0.16.0 are doing quite well? You want to prove that you can build OpenSSL via script?

@frankmorgner
Copy link
Member

It's simply impractical. There are too many possible locations. Even Apple still ships a deprecated version of OpenSSL (without advertising it).

With all those disk sucking magic tools that apple ships with OS X, I doubt that apple users care for some kilobytes more or less.

@mouse07410
Copy link
Contributor

mouse07410 commented May 17, 2016

It's simply impractical.

What "it"? Impractical to locate? Or impractical to document that users who want to use OpenSC must install a more-or-less recent version of OpenSSL, say 1.0.2g+ if they want to use libp11 and OpenSC, or 1.0.1+ if they want to use OpenSC?

I say - just document this requirement, and be done with it. Simple, effective, and non-invasive.

There are too many possible locations.

Fine - so let the user specify two directories, one for libcrypto.dylib and one for the *.h files. What seems to be the problem? Which somehow appears non-existent on my machines...

Even Apple still ships a deprecated version of OpenSSL (without advertising it).

So?

With all those disk sucking magic tools that apple ships with OS X, I doubt that apple users care for some kilobytes more or less.

And you're dead wrong here - I for one worry about every extra kilobyte (laptops, and unable to put 1TB drive to alleviate disk space concerns). Especially so because I must install OS updates, which - as you pointed out - already suck a lot of disk space.

@frankmorgner
Copy link
Member

And by the way, it's not just impractical. It's also improbable to find OpenSSL, because most users don't have OpenSSL installed via ports.

@mouse07410
Copy link
Contributor

mouse07410 commented May 17, 2016

Some install from the source. Some - via Macports. Some - via brew. Some - via Fink. Regardless, unless they stick to the Apple-provided apps, they are likely to use some kind of package management software, and among the stuff they installed, at least one package that needs OpenSSL is bound to occur. Which means - they'd have OpenSSL installed anyway.

If you think a user is capable of building OpenSC from the source, why wouldn't that same user be able to install OpenSSL in the very same way?

It's also improbable to find OpenSSL, because most users don't have OpenSSL installed via ports.

So let the user tell you where the relevant files are. Why is this so difficult???

@frankmorgner
Copy link
Member

this kind of detection at compile time is still possible. However, you can't expect users to have OpenSSL installed when creating binaries for distribution (i.e. the dmg everyone downloads). And this is actually what the script in question is intended to build. If you want to do something special then you're encouraged to tweak every step of the build process to your needs.

@mouse07410
Copy link
Contributor

mouse07410 commented May 17, 2016

However, you can't expect users to have OpenSSL installed when creating binaries for distribution (i.e. the dmg everyone downloads).

Well, I can - but I agree that the locations (and possibly versions) could be different from what the "customers" have.

If you want to do something special then you're encouraged to tweak every step of the build process to your needs.

I would much rather have a script that is easy to tweak, preferably in as small a number of places as possible. So for example, if your OpenSSL build block could just be commented out and replaced by

OPENSSL_CFLAGS=-I/opt/local/whatever -maes -mwhatever
OPENSSL_LIBS=-L/opt/local/wherever -lcrypto

I'd be happy enough. I don't want to interfere with what distribution builders need to do - but I don't want their process to make my simple re-build from the source a torture.

@frankmorgner
Copy link
Member

@mouse07410 see #764 for the OS X problems discussed

@mouse07410
Copy link
Contributor

mouse07410 commented May 19, 2016

@mouse07410 see #764 for the OS X problems discussed

I see no discussion there, just a PR that you say you validated on OS X. I cannot validate it, because my setup does not allow for pulling other versions of OpenSSL even if I wanted to (which I don't).

Again, as I do not produce DMGs for others, I don't feel I have a voice in the discussion about how the developers/maintainers should do the build.

My only need and hope is that my builds would still work, i.e., without bringing a different OpenSSL version to my machines. So as long as I can easily edit the script, commenting out the part that pulls and builds OpenSSL and specifying where my OpenSSL lives (without negative consequences to the rest of the build, like hidden dependencies and what not) - I cannot object.

Please make sure environmental variables OPENSSL_CFLAGS and OPENSSL_LIBS are respected throughout the build, and nothing else is needed to locate and access OpenSSL include files and libraries by the build process.

@viktorTarasov
Copy link
Member Author

So, what is the conclusion?

I have nothing to say about MacOS.
Afais, @frankmorgner and @pyther are for integrating the #764 to release,
and @mouse07410 is not categorically against.

What shall we do?

@frankmorgner
Copy link
Member

So no objections 😉; then I'll merge the PR. I'll also cherry-pick #763

@frankmorgner
Copy link
Member

There is also #759, which should be integrated into the release

@viktorTarasov
Copy link
Member Author

Yes, I'm looking on it. I will cherry-pick it.

@viktorTarasov
Copy link
Member Author

@frankmorgner I will wait for your merge and then will push few cherry-picked evident PRs.

@viktorTarasov
Copy link
Member Author

And it has to be a version that is ready to be tagged as official 0.16.0.

@frankmorgner
Copy link
Member

@viktorTarasov done

viktorTarasov added a commit that referenced this issue May 23, 2016
@mouse07410 has asked for it in
#688 (comment)

VTA: I do not see the difference (if the other arguments are properly used),
but assume that @mouse07410 has it's own valid reasons

Also included the few coding style touches.
@viktorTarasov
Copy link
Member Author

@frankmorgner fine, thanks.

@mouse07410
Copy link
Contributor

mouse07410 commented May 23, 2016

I've tested the current Github version of towards-0.16.0, and it passed all my tests. I had to make one change though:

diff --git a/MacOSX/build-package.in b/MacOSX/build-package.in
index 55b15fe..e0365ea 100755
--- a/MacOSX/build-package.in
+++ b/MacOSX/build-package.in
@@ -19,7 +19,7 @@ export CFLAGS="$CFLAGS -isysroot $SDK_PATH -arch x86_64 -mmacosx-version-min=10.

 export SED=/usr/bin/sed
 PREFIX=/Library/OpenSC
-export PKG_CONFIG_PATH=/usr/lib/pkgconfig
+export PKG_CONFIG_PATH=/opt/local/lib/pkgconfig

 if ! pkg-config libcrypto --atleast-version=1.0.1; then
        # OpenSSL is not installed

The reason is obvious: it is well-known that Apple's version of OpenSSL is beneath contempt, so it's pretty much pointless to confirm that it's there and that it's old.

What I need is for OpenSC to find the version that I installed. Which happens to live in /opt/local (which, BTW, is the only location of pkg-config binary - it looks like OSX does not include it).

Please don't mess with this env variable at all: if the system has pkg-config, it must know its default path; and if the system does not have pkg-config, then what's the point...

@lbevilacqua
Copy link

Hello there,
I built the Mac flavor of towards-0.16.0 and I noticed one thing: is it normal/known that Keychain Access doesn't display anymore the certificates inside my italian CNS-like cards (but can lock and unlock them)?
I tried to login on some website with the card (eg. university, public administration services, etc) and Safari can "see" the certificates inside the card (and Firefox too using opensc-pkcs11.so)...

@mouse07410
Copy link
Contributor

mouse07410 commented May 29, 2016

Showstopper Fixed by 9d88ae3

Doing regression testing on my old scripts, I found that pkcs11-tool no longer can derive EC keys. See #771

@mouse07410
Copy link
Contributor

@lbevilacqua you may need a working tokend.

@lbevilacqua
Copy link

@mouse07410 Thank you for pointing that out, I guess the government-provided software messed with the OpenSC installation. Installing RC2 on a clean system reconized 3 different cards without any issue.

@viktorTarasov
Copy link
Member Author

OpenSC 0.16.0 is created, thank you for your time and your efforts.

@frankmorgner
Copy link
Member

I also attached an image for OS X 10.10 and later

@viktorTarasov
Copy link
Member Author

Fine, I pushed the image to souceforge.

@CardContact
Copy link
Member

Thanks Viktor and all the other contributors for making this release happen.

Andreas

On 06/03/2016 02:45 PM, viktorTarasov wrote:

OpenSC 0.16.0 is created, thank you for your time and your efforts.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#688 (comment)

---------    CardContact Systems GmbH

|.##> <##.| Schülerweg 38
|# #| D-32429 Minden, Germany
|# #| Phone +49 571 56149
|'##> <##'| http:https://www.cardcontact.de
--------- Registergericht Bad Oeynhausen HRB 14880
Geschäftsführer Andreas Schwier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants